SYSTEM AND A METHOD FOR LEARNING FEATURES ON GEOMETRIC DOMAINS

Abstract

A method for extracting hierarchical features from data defined on a geometric domain is provided. The method includes applying on said data at least an intrinsic convolution layer, including the steps of applying a patch operator to extract a local representation of the input data around a point on the geometric domain and outputting the correlation of a patch resulting from the extraction with a plurality of templates. A system to implement the method is also described.

Claims

1. A method for extracting hierarchical features from data defined on a geometric domain, comprising applying on said data at least an intrinsic convolution layer, the method including the steps of: applying a patch operator to extract a local representation of the input data around a point on the geometric domain and outputting the correlation of a patch resulting from the extraction with a plurality of templates.

2. The method according to claim 1, further including the steps of: defining a local system of multi-dimensional pseudo-coordinates around a point on the geometric domain; computing a plurality of weighting functions acting on said pseudo-coordinates; applying said weighting functions to define said patch operator.

3. The method according to claim 1, further applying at least one of the following layers: a linear layer, including outputting a weighted linear combination of input data; a non-linear layer, including applying a non-linear function to input data; a spatial pooling layer, including: determining a subset of points on the geometric domain; for each point of said subset, determining the neighbours on the geometric domain; and computing an aggregation operation on input data over the neighbours for all the points of said subset; a covariance layer, including computing a covariance matrix of input data over all the points of the geometric; a fully connected layer, including outputting a weighted linear combination of input data at all the points of the geometric domain, wherein each layer has input data and output data and output data of one layer are given as input data to another layer.

4. The method according to claim 3, wherein said spatial pooling layer comprises aggregating of input data over the whole domain.

5. The method according to claim 3, wherein said aggregation operation of input data comprises one of the following: maximum computation; average computation; weighted average computation; average of squares computation.

6. The method according to claim 3, wherein two or more of said layers are applied in sequence, and the output data of one layer in the sequence is given as input data to a subsequent layer in the sequence.

7. The method according to claim 3, wherein more than one of said layers is applied and wherein parameters of the applied layers comprise one or more of the following: weights of the linear layers; templates of the intrinsic convolutional layers; parameters of the weighting functions used to compute the patch operators in the intrinsic convolutional layer.

8. The method according to claim 7, wherein parameters of the applied layers are determined by minimizing a cost function by means of an optimization procedure.

9. The method according to claim 8, wherein a plurality of said cost functions are defined and wherein each of said cost functions is associated to one or more application for which feature extraction is carried out.

10. The method according to claim 8, wherein the input into the optimization procedure is a training set comprising: a positive set of pairs of points on one or more geometric domains a negative set of pairs of points on one or more geometric domains; and wherein two identical copies of the hierarchical system configured with the same parameters are fed with pairs of points from said positive and negative sets, and wherein the optimization procedure tries to minimize the distance between the outputs corresponding to positive pairs and maximize the distance between the outputs corresponding to negative pairs.

11. The method according to claim 1, wherein the geometric domain is one of the following: a manifold; a parametric surface; an implicit surface; a mesh; a point cloud; an undirected weighted or unweighted graph; a directed weighted or unweighted graph.

12. The method according to claim 2, wherein the multi-dimensional pseudo-coordinates have one or more dimensions and comprise one or more of the following: geodesic coordinates; diffusion coordinates; intrinsic polar coordinates; vertex degree.

13. The method according to claim 2, wherein said weighting functions are fixed functions.

14. The method according to claim 2, wherein said weighting functions are parametric functions.

15. The method according to claim 2, wherein said weighting functions are one of the following: Gaussian kernels; spline kernels; trigonometric functions; orthogonal basis functions.

16. The method according to claim 14, wherein said weighting functions are sums of scaled trigonometric functions, and said parameters comprise the scale factors of the trigonometric functions.

17. The method according to claim 14, wherein said weighting functions are sums of scaled Gaussian kernels, and where said parameters comprise: the scale factors of the Gaussian kernels; the mean vectors of the Gaussian kernels, or a subset of elements thereof; the covariance matrices of the Gaussian kernels, or a subset of elements thereof.

18. The method according to claim 17, wherein the covariance matrices of the Gaussian kernels are diagonal, and the subset of their elements comprises the diagonal elements.

19. The method according to claim 1, wherein the patch operator inputs data on geometric domain and said point on said domain, and outputs the local representation of said data around said point, wherein the local representation is one or more of the following: data represented in a local intrinsic polar system of coordinates; data transformed by a windowed Fourier transform; data weighted by anisotropic diffusion kernels.

20. The method according to claim 19, wherein the patch operator outputs said local representation of input data in the local intrinsic polar system of coordinates, and wherein an origin of angular coordinates of the local intrinsic polar system of coordinates is provided as side information extracted from the geometric domain or the data.

21. The method according to claim 20, wherein the geometric domain is a surface and the side information used to determine the origin of the angular coordinate of the local intrinsic polar system of coordinates at each point on the surface is a principle curvature direction at said point.

22. The method according to claim 20, wherein the side information used to determine the origin of the angular coordinate of the local intrinsic polar system of coordinates at each point is a direction of a minimal or maximal absolute change of the data at said point.

23. The method according to claim 19, wherein the patch operator outputs said local representation of input data in the local intrinsic polar system of coordinates, and the Fourier transform is applied with respect to angular coordinates, followed by an absolute value operation.

24. The method according to claim 19, wherein the computation of said correlation between said patch and said plurality of templates further comprises the steps of: rotating each template along angular coordinates; computing the correlation of the patch with the rotated template; taking a maximum operation over all the rotations.

25. The method according to claim 19, wherein the representation of input data in the local polar system of coordinates around the point on the geometric domain further comprises the steps of: computing an intrinsic distance from said point to all the other points on the geometric domain; computing level sets of said intrinsic distance at a plurality of levels; splitting a full angle at said point into a plurality of angular directions; shooting rays emanating from said point along said directions perpendicular to said level sets.

26. The method according to claim 25, wherein said intrinsic distance is one of the following or an approximation thereof: geodesic distance; diffusion distance; commute time distance; biharmonic distance.

27. The method according to claim 25, wherein the representation of input data further comprises the steps of: computing weights localized around level sets and rays; computing weighted averages of input data with each of said weights.

28. The method according to claim 19, wherein computing the windowed Fourier transform of input data further comprises: computing localized modulated atoms; computing inner products of data with said atoms.

29. The method according to claim 28, wherein localized modulated atoms are heat kernels multiplied by Laplacian eigenfunctions.

30. The method according to claim 29, wherein the heat kernels are anisotropic.

31. The method according to claim 19, wherein the computation of local representation of input data around a point on geometric domain further comprises the steps of: computing a plurality of directed anisotropic heat kernels at said point, corresponding to a plurality of anisotropy constants and directions; computing weighted averages of input data with said directed anisotropic heat kernels.

32. The method according to claim 2, wherein the points of the geometric domain are in a grid, and wherein said step of applying the intrinsic convolutional layer on input data comprises applying a sliding window operation.

33. The method according to claim 32, wherein the sliding window operation further comprises: determining the center point of the window; extracting a block of points around said center point of the window; defining said multi-dimensional pseudo-coordinates for each point of said block of points; computing said plurality of weighting functions acting on said multi-dimensional pseudo-coordinates; applying said weighting functions to define said patch operator extracting a local representation of the input data at the points of said block of points; computing the correlation of said patch resulting from the extraction with a plurality of templates; and moving the window to a next adjacent location.

34. The method according to claim 2, wherein the geometric domain is a directed graph, and further comprising the steps of: inputting a plurality of graph motifs; for each input graph motif computing a new undirected weighted graph wherein the vertices are those of the input graph, and each edge is weighted by the occurrence of the graph motif; computing said multi-dimensional pseudo-coordinates around each vertex of said undirected weighted graph; computing said plurality of weighting functions acting on said pseudo-coordinates; applying the weighting functions to define said patch operator extracting said local representation of the input data around said point on the geometric domain; and outputting the correlation of said patch resulting from the extraction with said plurality of templates.

35. The method according to claim 34, wherein the multi-dimensional pseudo-coordinates computed for each undirected weighted graph comprise at least one of the following: vertex degree; geodesic distance from a vertex; diffusion distance from a vertex.

36. The method according to claim 34, wherein the weighting functions are diffusion kernels computed on the weighted undirected graphs for all the input graph motifs.

37. The method according to claim 36, wherein the diffusion kernels comprise heat kernels with a plurality of time scales.

38. The method according to claim 37, wherein the diffusion kernels are anisotropic diffusion kernels.

39. The computer implemented method according to claim 1 for extracting hierarchical features from data defined on a geometric domain, wherein said data are defined and stored in a storage of a computer, said computer including means for applying on said data at least an intrinsic convolution layer, said means executing the steps of: applying a patch operator to extract a local representation of the input data around a point on the geometric domain and outputting the correlation of a patch resulting from the extraction with a plurality of templates.

40. The method according to claim 1, including extracting hierarchical features from data defined on another geometric domain, applying on data defined on said another geometric domain at least the intrinsic convolution layer, including the steps of: applying a patch operator to extract a local representation of further input data around a point on the another geometric domain and outputting the correlation of a patch resulting from the extraction of the local representation of the further input data with said plurality of templates, wherein data defined on the geometric domain are associated to a first geometric object and said data are defined on the additional geometric domain are associated to a second geometric object, and wherein a similarity of the first object to the second object is measured based on the hierarchical features extracted from data defined on the geometric object and the hierarchical features extracted from data defined on said another geometric object.

41. The method according to claim 40, wherein said similarity measure is used for a shape retrieval application of the first object, wherein data defined on a plurality of geometric domains are provided as a collection of second objects, a similarity between each second object of the collection and the first object is measured, an object of the collection having the highest measure of similarity is returned as an object retrieved in the first object.

42. The method according to claim 40, wherein the geometric domain is a graph modelling a social network wherein the first object and second object are nodes of the graph corresponding to members of the social network, and wherein said similarity measure is a measure of similarity between said members of the social network, and wherein the measure of similarity is used to estimate a behaviour of one of the member in the social network based on a known behaviour of the other member of the social network with which the similarity has been measured, wherein said estimated behaviour is considered to be the known behaviour for a value of said measured similarity which is above a predetermined threshold.

43. The method according to claim 1, wherein said extracted hierarchical features are used for determining correspondences between objects from a class of geometric objects, provided as a plurality of data defined on a corresponding plurality of geometric domains, and an first object associated to said data defined on the geometric domain.

44. The method according to claim 43, wherein said correspondences between said class of geometric objects and said first object are used to automatically transfer a texture of said first object to said class of objects.

45. The method according to claim 43, wherein said correspondences between said class of geometric objects and said first object are used to automatically transfer an animation of said reference object to said class of objects.

46. A computer system for extracting hierarchical features from data defined on a geometric domain, comprising means for applying on said data at least an intrinsic convolution layer, where said means are configured to: apply a patch operator for extracting a local representation of the input data around a point on the geometric domain and to return as output the correlation of a patch resulting from the extraction with a plurality of templates.

47. The system according to claim 46, further including means to define said the operator, said means to define the patch operator being configured to: define a local system of multi-dimensional pseudo-coordinates around a point on the geometric domain; compute a plurality of weighting functions acting on said pseudo-coordinates; applying said weighting functions to define the patch operator.

48. The system according to claim 46, further including means for applying at least another layer among the following layers: a linear layer, including outputting a weighted linear combination of input data; a non-linear layer, including applying a non-linear function to input data; a spatial pooling layer, wherein: a subset of points on the geometric domain are determined; for each point of said subset, the neighbours on the geometric domain are determined; and an aggregation operation on input data over the neighbours for all the points of said subset is computed; a covariance layer, wherein a covariance matrix of input data over all the points of the geometric domain is computed; a fully connected layer, having as output a weighted linear combination of input data at all the points of the geometric domain, wherein each layer has input data and output data and said system comprehends means that are configured to provide as input data to a layer the output data of another layer.

49. The system according to claim 48, wherein said spatial pooling layer comprises an aggregation of input data over the whole domain.

50. The system according to claim 48, wherein said means to apply the spatial pooling layer are configured to process said aggregation of input data by means of one of the following computation: maximum computation; average computation; weighted average computation; average of squares computation.

51. The system according to claim 48, wherein said means for applying at least another layer are configured to apply at least another layer in sequence, providing the output data of one layer in the sequence as input data to a subsequent layer in the sequence.

52. The system according to claim 48, wherein said means for applying the intrinsic convolutional layer and said means for applying at least another layer are configured to set one or more of the following parameters to the applied layers: weights of the linear layers; templates of the intrinsic convolutional layers; parameters of the weighting functions used to compute the patch operators in the intrinsic convolutional layer.

53. The system according to claim 52, wherein said means for applying the intrinsic convolutional layer and said means for applying at least another layer are configured to determine parameters of the applied layers by minimizing a cost function by means of an optimization procedure.

54. The system according to claim 53, wherein a plurality of said cost functions are defined and wherein each of said cost functions is associated to one or more application for which feature extraction may be carried out by the system.

55. The system according to claim 53, wherein the input into the optimization procedure is a training set comprising: a positive set of pairs of points on one or more geometric domains; and a negative set of pairs of points on one or more geometric domains; and wherein two identical copies of the hierarchical system configured with the same parameters are fed with pairs of points from said positive and negative sets, and wherein the optimization procedure is configured to minimize the distance between the outputs corresponding to positive pairs and maximize the distance between the outputs corresponding to negative pairs.

56. The system according to claim 46, wherein the geometric domain is one of the following: a manifold; a parametric surface; an implicit surface; a mesh; a point cloud; an undirected weighted or unweighted graph; a directed weighted or unweighted graph.

57. The system according to claim 47, wherein the multi-dimensional pseudo-coordinates have one or more dimensions and comprise one or more of the following: geodesic coordinates; diffusion coordinates; intrinsic polar coordinates; vertex degree.

58. The system according to claim 47, wherein said weighting functions are fixed functions.

59. The system according to claim 47, wherein said weighting functions are parametric functions.

60. The system according to claim 47, wherein said weighting functions are one of the following: Gaussian kernels; spline kernels; trigonometric functions; orthogonal basis functions.

61. The system according to claim 59, wherein said weighting functions are sums of scaled trigonometric functions, and said parameters comprise the scale factors of the trigonometric functions.

62. The system according to claim 59, wherein said weighting functions are sums of scaled Gaussian kernels, and wherein said parameters comprise: the scale factors of the Gaussian kernels; the mean vectors of the Gaussian kernels, or a subset of elements thereof; the covariance matrices of the Gaussian kernels, or a subset of elements thereof.

63. The system according to claim 62, wherein the covariance matrices of the Gaussian kernels are diagonal, and the subset of their elements comprises the diagonal elements.

64. The system according to claim 46, wherein the patch operator is configured to input data on geometric domain and said point on said domain, and to output the local representation of said data around said point, wherein the local representation is one or more of the following: data represented in a local intrinsic polar system of coordinates; data transformed by a windowed Fourier transform; data weighted by anisotropic diffusion kernels.

65. The system according to claim 64, wherein the patch operator is configured to output said local representation of input data in the local intrinsic polar system of coordinates, and wherein to provide an origin of angular coordinates of the local intrinsic polar system of coordinates as side information extracted from the geometric domain or the data.

66. The system according to claim 65, wherein the geometric domain is a surface and the side information for determining the origin of the angular coordinate of the local intrinsic polar system of coordinates at each point on the surface is a principle curvature direction at said point.

67. The system according to claim 65, wherein the side information for determining the origin of the angular coordinate of the local intrinsic polar system of coordinates at each point is a direction of a minimal or maximal absolute change of the data at said point.

68. The system according to claim 64, wherein the patch operator is configured to output said local representation of input data in the local intrinsic polar system of coordinates, and to apply the Fourier transform with respect to angular coordinates, followed by an absolute value operation.

69. The system according to claim 64, configured to compute the correlation between said patch and said plurality of templates by: rotating each template along angular coordinates; computing the correlation of the patch with the rotated template; taking a maximum operation over all the rotations.

70. The system according to claim 64, wherein the patch operator if further configured to represent input data in the local polar system of coordinates around the point on the geometric domain by: computing an intrinsic distance from said point to all the other points on the geometric domain; computing level sets of said intrinsic distance at a plurality of levels; splitting a full angle at said point into a plurality of angular directions; shooting rays emanating from said point along said directions perpendicular to said level sets.

71. The system according to claim 70, wherein said intrinsic distance is one of the following or an approximation thereof: geodesic distance; diffusion distance; commute time distance; biharmonic distance.

72. The system according to claim 70, wherein the patch operator if further configured to represent input data by: computing weights localized around level sets and rays; computing weighted averages of input data with each of said weights.

73. The system according to claim 64, wherein the patch operator is further configured to compute the windowed Fourier transform of input data by: computing localized modulated atoms; computing inner products of data with said atoms.

74. The system according to claim 73, wherein localized modulated atoms are heat kernels multiplied by Laplacian eigenfunctions.

75. The system according to claim 74, wherein the heat kernels are anisotropic.

76. The system according to claim 64, further configured to compute the local representation of input data around a point on geometric domain by: computing a plurality of directed anisotropic heat kernels at said point, corresponding to a plurality of anisotropy constants and directions; computing weighted averages of input data with said directed anisotropic heat kernels.

77. The system according to claim 47, wherein the points of the geometric domain are in a grid, and wherein said means for applying the intrinsic convolutional layer on input data are configured to apply a sliding window operation.

78. The system according to claim 77, wherein said means for applying the intrinsic convolutional layer are further configured to: determining the center point of the window; extracting a block of points around said center point of the window; defining said multi-dimensional pseudo-coordinates for each point of said block of points; said plurality of weighting functions acting on said multi-dimensional pseudo-coordinates; said weighting functions to define said patch operator extracting a local representation of the input data at the points of said block of points; computing the correlation of said patch resulting from the extraction with a plurality of templates; and moving the window to a next adjacent location.

79. The system according to claim 47, wherein the geometric domain is a directed graph and wherein means to input graph motifs are configured to: input a plurality of graph motifs; for each input graph motif computing a new undirected weighted graph wherein the vertices are those of the input graph, and each edge is weighted by the occurrence of the graph motif; computing said multi-dimensional pseudo-coordinates around each vertex of said undirected weighted graph; computing said plurality of weighting functions acting on said pseudo-coordinates; applying the weighting functions to define said patch operator extracting said local representation of the input data around said point on the geometric domain; and outputting the correlation of said patch resulting from the extraction with said plurality of templates.

80. The system according to claim 79, wherein the multi-dimensional pseudo-coordinates of each undirected weighted graph comprise at least one of the following: vertex degree; geodesic distance from a vertex; diffusion distance from a vertex.

81. The system according to claim 79, wherein the weighting functions are diffusion kernels and wherein said means to define the operator are configured to compute the weighted undirected graphs for all the input graph motifs.

82. The system according to claim 81, wherein the diffusion kernels comprise heat kernels with a plurality of time scales.

83. The system according to claim 82, wherein the diffusion kernels are anisotropic diffusion kernels.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0143] FIG. 1 represents the application of 3D filters on a cylinder before and after a deformation, according to a method of the prior art.

[0144] FIG. 2 represents the inability of spectral convolutional networks proposed in prior art to generalize the model learned on one domain to another domain.

[0145] FIG. 3 represents the application of a patch on the cylinder of FIG. 1, according to the method of the present invention.

[0146] FIGS. 4A-4B schematically represents a geometric domain in the form of a 3D shape and a geometric domain in the form of a graph, respectively.

[0147] FIG. 5 schematically shows the terms involved in the definition of the LBO discretization.

[0148] FIG. 6A-6B schematically represents a discrete representation of a geometric domain in the form of a manifold triangular mesh and a discrete representation of a geometric domain in the form of a point cloud, respectively.

[0149] FIG. 7 schematically represents the translation and modulation operators for the definition of patch in the form of WFT analysis for two different windows, according to the method of the present invention.

[0150] FIG. 8 schematically shows a patch in the form of a local polar system of coordinates on a 3D shape, according to the method of the present invention.

[0151] FIGS. 9A and 9B schematically represents the radial interpolation weights and the angular interpolation weights, according to the method of the present invention.

[0152] FIGS. 10A-10F schematically represent the patch discretization when the patch is in the form of the local polar system of coordinates, according to the method of the present invention.

[0153] FIG. 11A schematically represents the patch of FIG. 8 separately from the 3D shape, and FIG. 11B schematically represents the patch of FIG. 11A after a conversion into a form adapted to be correlated with a template, according to the method of the present invention.

[0154] FIG. 12 schematically represents the patch of FIG. 11B wherein different values for each bin are represented with different patterns, according to the method of the present invention.

[0155] FIG. 13 schematically represents a comparison between isotropic heat kernels and a variety of anisotropic heat kernels for different anisotropy intensities and orientations, where the geometric domain is a mesh, according to the method of the present invention.

[0156] FIG. 14 schematically represents a comparison between isotropic heat kernels and a variety of anisotropic heat kernels for different anisotropy intensities and orientations, where the geometric domain is a point cloud, according to the method of the present invention.

[0157] FIG. 15 schematically shows the terms involved in the definition of the anisotropic LBO discretization.

[0158] FIG. 16 schematically shows the radial and angular components of the local system of intrinsic polar coordinates, according to the method of the present invention.

[0159] FIG. 17 schematically shows different families of weighting functions generating different patch operators considered as particular instances of the present invention.

[0160] FIG. 18 schematically shows the sliding window definition on the discretization of a raster scan with a regular grid.

[0161] FIG. 19 shows various examples of graph motifs for a basic graph consisting of 3 vertices only.

DETAILED DESCRIPTION

[0162] According to the idea of solution given above, a method is hereafter described for extracting hierarchical features from data defined on a geometric domain using an intrinsic convolutional neural network endowed with, at least, one intrinsic convolution layer, according to the present invention. The method is described with reference to the annexed drawings, given just for exemplification purpose and without limiting the scope of protection of the application.

[0163] The intrinsic convolution layer of the method of the present invention differs from the convolution layer of convolutional neural networks known in the prior art by the kind of domains onto which it is applicable. In particular, the convolution layer of prior art neural network is restricted to domains that can be modelled as Euclidean spaces (images, audio signals, etc.). Advantageously, the intrinsic convolution layer of the method of the present invention is adapted to deal with a wider category of domains that can be modelled as non-Euclidean spaces.

[0164] Advantageously, the method of the proposed invention works at an intrinsic level: the patch operator according to the present invention is defined on the geometric domain itself, as opposed to prior art methods using a Euclidean representation (embedding) of non-Euclidean domains. For instance, FIG. 3 schematically shows such advantage in the case where the geometric domain is represented as a cylinder 1 and the deformation is non-rigid: the patch 7 around a point 2 on the cylinder 1 follows the deformation, making the patch invariant to such deformations. In other words, the patch of the present invention is intrinsic.

[0165] The method according to the present invention is implemented in a computer system, i.e. the invention relates to a computer implemented method. The computer system may include a network. More particularly, any processing step is executed in means of said computer system, and any data in input to the method or provided as output thereof is stored in a memory of the computer system.

[0166] Hereafter, definitions of geometric domain, data defined on the geometric domain, intrinsic convolutional layer, patch operator, correlation with templates, local system of pseudo-coordinates, weighting functions and intrinsic convolutional neural network according to the method of the present invention are given.

[0167] Geometric Domain

[0168] A geometric domain is a non-Euclidean space. Therefore, the meaning of the two expressions in the following description is the same. Geometric domains include, but are not limited to, manifolds and graphs. FIG. 4A shows an example of a manifold, while FIG. 4B shows an example of a graph.

[0169] A M-dimensional manifold X is a topological space that is locally homeomorphic (topologically equivalent) to a M-dimensional Euclidean space, referred to as the tangent space. For example, the Earth (spherical surface) is locally equivalent to a plane. Additionally, the manifold X can be endowed with an inner product on the tangent space (referred to as Riemannian metric), which provides a way to measure intrinsic distances on the manifold. In this case X is referred to as a Riemannian manifold.

[0170] What is commonly known as a three-dimensional (3D) shape in Computer Graphics and Vision can be modelled as two-dimensional (2D) manifold (surface), embedded in the 3D Euclidean space. The term 2D in this case refers to the intrinsic dimensionality of the manifold, while the term 3D refers to the (extrinsic) dimensionality of the embedding space.

[0171] Under some conditions, a M-dimensional manifold embedded in a D-dimensional Euclidean space (D>M) can be represented through a parameterization y=(x.sub.1, . . . , x.sub.M), : .Math..sup.M.fwdarw..sup.D, where represents the parameter space. In the case when M=2, y=(x.sub.1, x.sub.2) is referred to as parametric surface. A relevant example of parametric surface is represented by raster scans, the kind of data acquired by 3D cameras such as Microsoft Kinect or Intel RealSense. Raster scans are parametric surfaces y=(x.sub.1, x.sub.2), where the function captures the depth y of the point (x.sub.1, x.sub.2).

[0172] Another way of representing a M-dimensional manifold is through an implicit form as the M-dimensional level set of a function (x.sub.1, . . . , x.sub.D)=0. For example, a 2-dimensional surface can be represented as the level set of a function (x.sub.1, x.sub.2, x.sub.3)=0. A common choice for is the signed distance function of the 3D coordinate (x.sub.1, x.sub.2, x.sub.3) from the surface.

[0173] Geometric Domain Discretization

[0174] According to the present invention, the geometric domain defined in terms of one of the previous formulas is associated to a discrete approximation in order to be processed by the computer implemented method of the present invention.

[0175] For instance, the manifold X can be associated to a discrete approximation X by sampling N points x.sub.1, . . . , x.sub.N from X. The discrete approximation can be represented in different ways, including: [0176] a point cloud V.sup.N3 where for each point x.sub.i, i=1, . . . , N, is specified a coordinate in .sup.3 stored in the ith row of the matrix V; [0177] a triangular mesh (V,E,F), with vertices V.sup.N3 edges E{1, . . . , N}{1, . . . , N}, and faces F{1, . . . , N}{1, . . . , N}{1, . . . , N}. A manifold triangular mesh is a triangular mesh where each interior edge (i,j)E is shared by exactly two triangular faces (i,j,k), (i,j,h)F, as represented in FIG. 5, and boundary edges belongs to exactly one triangular face.

[0178] FIG. 6A schematically represents a discrete approximation of a geometric domain X in the form of a point cloud, and FIG. 6B a discrete approximation of the geometric domain X in the form of a manifold triangular mesh.

[0179] Graphs, instead, can be modelled as the pair (V,E), where V={1, . . . , N} is the set of N vertices or nodes, and E{1, . . . , N}{1, . . . , N} is the set of edges connecting the vertices. The main difference with the previous models resides in the fact that a graph is a purely topological notion, therefore the vertices does not need to be associated with any coordinate in .sup.3. A weighted graph is a graph where each edge (i,j) is associated with some scalar weight w.sub.ij. If such weights are not specified, the graph is called unweighted. A directed graph is a graph whose edges has a direction associated with them, i.e. the edge (i,j) is different form the edge (j,i). If such direction is not specified, the graph is called undirected.

[0180] Data Defined on Geometric Domain

[0181] According to the present invention, given the geometric domain X, data on X are defined in terms of functions : X.fwdarw..sup.Q, where Q represents the dimensionality of the data. When the geometric domain X is approximated with one of the previous discrete approximations X, data on X can be defined in terms of maps f: V.fwdarw..sup.Q, where V are the vertices of the discretization X. If Q=1, than f is represented by the N-dimensional vector f=((x.sub.1), . . . , (x.sub.N)).sup.T, where .sup..T is the transposition operator.

[0182] The steps of the method of the present invention are executed on data defined on the discrete approximation X of the geometric domain X.

[0183] Examples of such data comprises: [0184] geometric information, such as the Gaussian curvature K (x.sub.i)=.sub.m(x.sub.i) .sub.M(x.sub.i), where .sub.m(x.sub.i), .sub.M(x.sub.i) represent the minimum and maximum curvature of the point x.sub.iX respectively; [0185] color information, such as color texture in the RGB color space f: V.fwdarw..sup.3, x.sub.i.fwdarw.(R.sub.i,G.sub.i,B.sub.i) where R.sub.i, G.sub.i, B.sub.i are scalars associated to the red, green and blue channels respectively; [0186] semantic information, such as the meaning or the name of some parts of the shape. If, for instance, we consider the 3D shape of a human body, the function f: V.Math.V.fwdarw. can map a sparse set of vertices V to their anatomical counterpart (knee, shoulder, . . . ) by associating to the vertex x.sub.iV a natural number as a label corresponding to the anatomical part. If we model a social network with a graph, examples of semantic information can be the genre of the people corresponding to the nodes (: V.fwdarw.{0, 1}, where 0 corresponds to male and 1 to female) or their favorite film (: V.fwdarw., where the films are labeled with natural numbers) or more generally, any function : V.fwdarw..

[0187] Data on the geometric domain X may be defined also through spectral properties of the Laplace-Beltrami operator (LBO) .sub.X=div.sub.X(.sub.X ), a generalization of the classical Laplacian to non-Euclidean spaces. The LBO is intrinsic, i.e. it does not depend on the embedding of the manifold. As a result, it is invariant to isometric (metric preserving) deformations of the manifold. If, for instance, the manifold X represents a human body, then the LBO and all the quantities derived from it are invariant to e.g. pose changes (different positions of arms, legs, . . . ). On a compact manifold, the LBO admits an eigendecomposition .sub.Xk=.sub.k.sub.k with real eigenvalues {0=.sub.1.sub.2 . . . }. The set of the eigenvalues {.sub.k}.sub.k1 is also known as the spectrum of the LBO. The corresponding eigenfunctions {.sub.k}.sub.k1, form an orthonormal basis on L.sup.2(X), which is a generalization of the Fourier basis to non-Euclidean domains.

[0188] Any function L.sup.2(X) can be represented as the Fourier series (x)=.sub.k1,.sub.k.sub.L.sub.2.sub.(X).sub.k(x), where the analysis =,.sub.k.sub.L.sub.2.sub.(X) can be thought as the forward Fourier transform and the synthesis .sub.k1 .sub.k(x) represents the inverse one. The eigenvalues {.sub.k}.sub.k1 plays the role of frequencies.

[0189] The generalized convolution of , gL.sup.2(X) can be defined by analogy to the classical case as the inverse transform of the product of forward transforms, (*g)(x)=.sub.k1{circumflex over (f)}.sub.k.sub.k .sub.k(x) and is, in general, non shift-invariant.

[0190] The LBO can be used to describe physical processes on geometric domains. For instance, the heat diffusion on a geometric domain X can be described through the isotropic heat equation:

[00001]$\{\begin{array}{c}{f}_t\left(x,t\right)={}_X.Math.f\left(x,t\right),\\ f\left(x,0\right)=f_0\left(x\right),\end{array}$

[0191] where (x,t) denotes the amount of heat at point x and time t and .sub.0(x) is the initial heat distribution. The solution of the previous equation can be obtained by convolution between the initial condition .sub.0 and the heat kernel h.sub.t(x,y), i.e. (x,t)=h.sub.t(x,y)*.sub.0(x). In the spectral domain, the heat kernel has the closed form expression h.sub.t(x,y)=.sub.k1 e.sup.t.sup.k .sub.k(x).sub.k(y).

[0192] The isotropic heat equation assumes that the heat conduction properties of the manifold are constant at every point. A more general diffusion equation has the form .sub.t(x,t)=div.sub.X(x).sub.X (x,t)), where D(x) is the thermal conductivity tensor (in the discrete settings, the operator D (x) can be represented as the 22 matrix D) applied to the gradient in the tangent plane. The thermal conductivity tensor allows modelling heat flow that is position- and direction-dependent; the corresponding heat diffusion equation in this case is called anisotropic. The eigendecomposition and the heat kernel associated with the anisotropic LBO .sub.X=div.sub.X(D(x) .sub.X(x,t)) are defined in the same way as the ones for the LBO one. We will refer to them as anisotropic eigenfunctions and anisotropic heat kernel respectively.

[0193] On a discrete approximation X of the geometric domain X, the LBO .sub.X can be defined as an NN matrix L=A.sup.1W, where

[00002]$w_{\mathrm{ij}}=\{\begin{array}{cc}\frac{\cot .Math..Math.{}_{\mathrm{ij}}+\cot .Math..Math.{}_{\mathrm{ij}}}{2}& \left(i,j\right)E\\ -{.Math.}_{ki}.Math.w_{\mathrm{ik}},& i=j\\ 0,& \mathrm{else}\end{array}$

[0194] where .sub.ij, .sub.ij denotes the angles ikj, ihj of the triangles sharing the edge ij, and A=diag(a.sub.1, . . . , a.sub.n) with

[00003]$a_i=\frac{1}{3}.Math.{.Math.}_{\mathrm{jk}:\mathrm{ijk}F}.Math.A_{\mathrm{ijk}}$

being the local area element at vertex i and A.sub.ijk denoting the area of the triangle ijk. FIG. 5 schematically shows the terms involved in the definition of the LBO discretization.

[0195] The first KN eigenvectors and eigenvalues of L are computed by performing the generalized eigendecomposition W=A, where =(.sub.1, . . . , .sub.k) is an NK matrix containing as columns the discretized eigenfunctions and =diag(.sub.1, . . . , .sub.k) is the diagonal matrix of the corresponding eigenvalues.

[0196] Through the eigendecomposition of the LBO it is possible to define meaningful features f: V.fwdarw..sup.Q, that can be considered as input data for the method of the present invention. A relevant category of them is represented by the known spectral descriptors. For instance, global point signature is defined as f(x)=(.sub.1.sup.1/2.sub.1(x), . . . , .sub.Q.sup.1/2.sub.Q(x)). More in general, spectral descriptors take the generic form of the diagonal of a parametric kernel diagonalized by the LBO eigenbasis. More specifically, at each point xX, a spectral descriptor can be constructed as

f(x)=.sub.k1(.sub.k).sub.k.sup.2(x),

[0197] where ()=(.sub.1(), . . . , .sub.Q()) represents a bank of transfer functions. By changing the transfer functions () different shape properties are described or emphasized. Relevant examples include: [0198] heat kernel signature (HKS) considers low-pass transfer functions .sub.t()=e.sup.t for various values of the parameter t{t.sub.1, . . . , t.sub.Q}. The resulting descriptor gives rise to the autodiffusivity function h.sub.t(x,x) (diagonal of the heat kernel), whose physical interpretation is the amount of heat remaining at x after time t; [0199] wave kernel signature (WKS) considers bans-pass transfer functions .sub.v()=e.sup.((log vlog )/2.sup.2) for various values of the parameter v={v.sub.1, . . . , v.sub.Q}. The physical interpretation of WKS is the probability to find a quantum particle at point x, given that it has an initial log-normal energy distribution with mean value v and variance ; [0200] geometry vector instead considers .sub.q()=.sub.q(.sub.k), q=1, . . . , Q, where ={.sub.1, . . . , .sub.Q} represents a B-spline basis on the spectrum of the LBO. Therefore each component f.sub.q of the descriptor, q=1, . . . , Q, looks like f.sub.q(x)=.sub.k1.sub.q(.sub.k).sub.k.sup.2(x).

[0201] Intrinsic Convolution Layer

[0202] According to the present invention, the convolution operation on the geometric domain X between said input data and a multitude of learnable templates g is defined by following the interpretation of convolution as correlation with a template. Said convolution operation and the relative convolution layer are referred to as intrinsic convolution and intrinsic convolution layer, to avoid ambiguities with prior art convolution operation and convolution layer, which are limited to be applied on Euclidean domains only and are not invariant to non-rigid deformations.

[0203] The intrinsic convolution layer comprises the steps of: [0204] extracting a local representation P(x) of the input data around a point xX. Such local representation is hereafter referred as patch and the operator P(x) that extract the patches from the geometric domain X is referred as patch operator; [0205] computing the correlation between the patches with a plurality of templates.

[0206] Patch Operators

[0207] According to an aspect of the present invention, the patch operator may be defined in different ways. In particular, in the following description, three different patch operators are defined through which, respectively: [0208] data can be transformed by a windowed Fourier transform; [0209] data can be weighted by Gaussians defined on a local polar system of coordinates; [0210] data can be weighted by anisotropic heat kernels.

[0211] The patch operator takes in input data on the geometric domain and a point on said domain, and outputs the local representation of said input data around said point, wherein the local representation, depending on which operator is considered, is data transformed by a windowed Fourier transform, data weighted by Gaussians on a local intrinsic polar system of coordinates, or data weighted by anisotropic diffusion kernels.

[0212] Advantageously, the patch operator to be used is selected depending on the kind of geometric domain on which data are defined. Moreover, depending on which patch operator is used, the step of correlating the patch with a template is adapted.

[0213] In the following, details of each of the three patch operators and how to correlate the extracted patches with a template are described.

[0214] Patch Operator Via Windowed Fourier Transform

[0215] On the base of the idea that convolution in the spatial domain corresponds to a multiplication in the frequency domain, the method of the present invention defines the patch operator by applying a vertex-frequency analysis in terms of a windowed Fourier transform (WFT) of the input data.

[0216] In such settings, the patch operator takes in input data on the geometric domain and a point on said domain, and outputs the local representation of said input data around said point by means of a WFT.

[0217] The computation of the patch operator using the WFT requires two steps: [0218] the computation of the localized modulated atoms; [0219] the computation of inner products with said atoms.

[0220] In the Euclidean domain, the classical WFT analyzes the frequency content of a signal that is localized by means of a local window h. Given a function L.sup.2(), and a window h localized at zero, the WFT is computed as (S)(x,)=(y)h(yx)e.sup.iydy. The WFT is a function of the spatial location of the window x and the modulation frequency . The choice of the window h allows to control the tradeoff between spatial and frequency localization, in the sense that narrower windows result in worse frequency resolution. The WFT can also be represented as an inner product with said translated and modulated window h, (S)(x,)=,M.sub.,T.sub.xh where T.sub.x represent the translation operator and M.sub. the modulation operator respectively.

[0221] The method of the present invention extends the notion of WFT to geometric domains by defining the translation operator as (T.sub.y)(x)=.sub.k1{circumflex over (f)}.sub.k.sub.k(y).sub.k(x) and by defining the modulation operator as (M.sub.k)(x)=.sub.k(x)(x), where {circumflex over (f)} is the Fourier transform of the input data and {.sub.k}.sub.k1 can be the isotropic LBO eigenfunctions or the anisotropic LBO eigenfunctions.

[0222] FIG. 7 schematically represents the translation and modulation operators for the definition of patch in the form of WFT analysis for two different windows 8a, 8b. In particular, with reference numbers: [0223] 9a (respectively, 9b) is indicated the translation operator centered around the point 13 and corresponding to the window 8a (respectively, 8b); [0224] 10a (respectively, 10b) is indicated the translation operator centered around the point 14 and corresponding to the windows 8a (respectively, 8b); [0225] 11a, 12a (respectively, 11b, 12b) are indicated modulation operators for the point 13 and corresponding to the window 8a (respectively, 8b);

[0226] where the values of the functions corresponding to the translation and modulation operator are represented as colors on the geometric domain.

[0227] Combining the two operators together, we obtain a modulated and translated window M.sub.kT.sub.xh, called atom. On geometric domains, the atoms can be defined as M.sub.kT.sub.xh=.sub.k(y).sub.k1.sub.k.sub.k(x).sub.k(y). Therefore, on geometric domains the WFT takes the form (S)(x,k)=,M.sub.kT.sub.xh.sub.L.sub.2.sub.(X)=.sub.l1.sub.l.sub.l(x),.sub.l.sub.k.sub.L.sub.2.sub.(X).

[0228] The patch operator corresponding to the point xX can be defined by collecting the output of the WFT corresponding to the first j=1, . . . , J frequencies, i.e. P.sub.j(x)=((S)(x,j), . . . , (S)(x,J)).

[0229] If a discrete approximation X of the geometric domain X is provided, the WFT can be computed as Sf=(f).sup.TA(.sup.T), where is the K-dimensional vector representing the window in the frequency domain, f is the N-dimensional vector representing the input function, A is the diagonal matrix containing the local area elements, and (aB).sub.ij=a.sub.iB.sub.ij denotes the known Hadamard product, i.e. element-wise multiplication of a vector and a matrix, replicating the vector along the second dimension of the matrix. The resulting WFT is a matrix of size KN.

[0230] If the patch operator is provided by means of the WFT, the intrinsic convolution operation can be defined as (*g)(x)=.sub.i=1.sup.I.sub.j=1.sup.Jg.sub.qij|(S.sub.i)(x,j)|, where f=(.sub.1(x), . . . , .sub.I(x)) represents the I-dimensional input function, q=1, . . . , Q corresponds to the output dimension, g is the learnable template and the absolute value is considered to reduce the effect of the LBO eigenfunctions sign ambiguities. In this way, the intrinsic convolution is reduced to a simple tensor multiplication.

[0231] Advantageously, according to our invention, the Fourier transform of the window can not only be a fixed filter, but can also be parameterized in some fixed interpolation basis in the frequency domain (e.g. the B-spline basis ={.sub.1, . . . , .sub.Q} on the LBO spectrum), e.g. .sub.i()=.sub.m=1.sup.Mb.sub.im.sub.m(), i=1, . . . , I, where the IM matrix (b.sub.im) of learnable weights defines the windows. In such case, the IM window parameters b.sub.im, i=1, . . . , I, m=1, . . . , M are considered additional parameters of the intrinsic convolution layer and can be optimized during the learning procedure.

[0232] Patch Operator Via Gaussians on a Local System of Intrinsic Polar Coordinates

[0233] According to an aspect of the proposed invention, the patch operator computes a local representation of the data around each input vertex of the geometric domain by interpolating the data with fixed Gaussian weights on a local system of intrinsic polar coordinates previously extracted.

[0234] More in details, given a point xX, such patch operator is constructed by the following steps: [0235] construction of a local system of intrinsic polar coordinates on the geometric domain X, i.e. a bijection (x):N(x)=B.sub.max(x).fwdarw.[0, .sub.max][0,2) that maps a local neighborhood N(x) of x to a system of intrinsic polar coordinates (, ). This step is agnostic on the data . FIG. 7 schematically shows how such local polar system of coordinates 16 around a point 15 looks on a 3D shape representing a man 17. [0236] construction of interpolation weights P(x) for the data on such polar system of coordinates, i.e. (P(x))(,)=(.sup.1(x))(m ). We will refer to P(x) as the patch operator and to P(x) as the patch on the geometric domain X centered at point xX.

[0237] The construction of the local system of intrinsic polar coordinates includes the following steps: [0238] extraction of intrinsic distance from the point xX to the points yN(x) in a neighbor of xX. Examples of intrinsic distances include: [0239] geodesic distance, which measures the length of the shortest path along the geometric domain X between two points x,yX. The main drawbacks of such distance are that it requires a mesh structure and it is sensitive to topological noise (e.g. holes in the geometric domains caused by an imprecise acquisition of the 3D geometry); [0240] diffusion distance, which measures the distance between two points xX, yN(x) by the diffusion process d.sub.X.sup.2(x,y)=.sub.k1e.sup.t.sup.k(.sub.k(x).sub.k(y)).sup.2, where .sub.k,.sub.k are the kth eigenfunction and eigenvalue of the LBO respectively; [0241] commute time distance, which measures the distance between two points xX, yN(x) as the average of the time a random walker would take to travel from x to y and back. The commute time distance can be represented in terms of eigenfunctions and eigenvalues of the LBO as

[00004]$d_X^2\left(x,y\right)={.Math.}_{k1}.Math.\frac{{\left({}_k\left(x\right)-{}_k\left(y\right)\right)}^2}{{}_k};$ [0242] biharmonic distance, measures the distance between two points x, yX as d.sub.X(x,y)=g(x,x)+g(y,y)2g(x,y), where

[00005]$\left(x,y\right)={.Math.}_{k1}.Math.\frac{{}_k\left(x\right).Math.{}_k\left(y\right)}{{}_k^2}$

is the Green's function of the biharmonic operator .sup.2. When represented in terms of eigenfunctions and eigenvalues of the LBO, the biharmonic distance formula differs from the commute time distance one only slightly:

[00006]$d_X^2\left(x,y\right)={.Math.}_{k1}.Math.\frac{{\left({}_k\left(x\right)-{}_k\left(y\right)\right)}^2}{{}_k^2};$ [0243] computation of the radial coordinates as the -level set {y:d.sub.X(x,y)=} of one of the previous intrinsic distances (geodesic, diffusion, . . . ) for =[0,.sub.max]. We call .sub.max the radius of the local patch. Empirically .sub.max can be chosen equal to some small percentage of the intrinsic distance diameter, e.g. .sub.max1% of the geodesic diameter; [0244] computation of the angular coordinates as a set of geodesics (x, ) emanating from x in direction such that they are perpendicular to the p-level set described in the previous step for =[0, 2). If instead, the origin of the angular coordinates of the local system of intrinsic polar coordinates is provided as a side information extracted from the geometric domain in term of a direction v, the angular coordinate measures the angle with respect to such given direction v. For instance, v can be chosen as the principal curvature direction of the geometric domain X in the point xX.

[0245] The previous steps are repeated for all the point of the geometric domain X. In particular, FIG. 8 shows the resulting patch when the steps above are executed starting from the point 15 and the geometric domain is a man 17.

[0246] Given the angular and radial coordinates (, ) corresponding to the point xX, the patch operator P (x) can be obtained as:

(P.sub.j(x))(,)=.sub.Xw.sub.j(,)(y)dy, j=1, . . . ,J,

[0247] where the interpolation weights w.sub.j can be defined as the Gaussians:

[00007]$w_j\left(,\right)=\frac{w_{}\left(x,y\right).Math.w_{}\left(x,y\right)}{{}_X.Math.w_{}\left(x,y\right).Math.w_{}\left(x,y\right).Math.\mathrm{dy}},$

[0248] where w.sub.(x,y), w.sub.(x,y) are the radial and angular interpolation weights respectively. For instance, the radial interpolation weights can be defined as a Gaussian w.sub.(x,y)e.sup.(d.sup.X.sup.(x,y)).sup.2.sup./.sup..sup.2 of the intrinsic distance from x, centered around the radius . FIG. 9A schematically represents the radial interpolation weights for the point 15 along the radius 18, where the weights are represented in different grey scales. The angular interpolation weights can be defined as a Gaussian w.sub.(x,y)e.sup.d.sup.X.sup.2.sup.((x,),y)/.sup..sup.2 the point-to-set distance d.sub.X.sup.2((x,),y)=min.sub.z(x,)d.sub.X(z,y), where (x,) represents a geodesic emanating from x in the direction . FIG. 9B schematically represents the angular interpolation weights for the point x along the angle 19, where the weights are represented in different grey scales. The previous steps are repeated for all the point of the geometric domain X.

[0249] If the geometric domain is approximated with a manifold triangular mesh, the local system of coordinates previously described is sampled at N.sub. angular and N.sub. radial bins, obtaining in this way a discrete local system of coordinates. More in details, with reference to FIGS. 10A-10F, the discretization procedure involves the following steps: [0250] start with a vertex 15, partition the 1-ring 20 of the vertex 15 by N.sub. rays 21 into equiangular bins, and align the first ray 21a with one of the edges 22, as illustrated in FIG. 10A; [0251] propagate ray 21b into adjacent triangle 23 using an unfolding procedure (similar to the known one used in fast marching algorithm; as illustrated in FIG. 10B-10F), thus producing the poly-lines 24 that form the angular bins; [0252] create radial bins as level sets of the intrinsic distance function. If the chosen intrinsic distance is the geodesic one, then it can be discretized using the fast marching algorithm.

[0253] As a consequence, the patch operator P(x) can be represented in the discrete domain as a N.sub.N.sub.NN matrix P applied to the data f on the mesh and producing the patches at each vertex. Conveniently, the matrix P is very sparse since only the values of the function at a few nearby vertices contribute to each local polar bin.

[0254] Once the patch operator P(x) is computed, the intrinsic convolution of L.sup.2(X) with a template g(,) is defined as (*g)(x)=.sub.0.sup.2.sub.0.sup..sup.maxg(,)(P(x))(,)dpd, where g(,) is a template (or filter) applied on the patch (i.e. expressed in the same local system of intrinsic polar coordinates than the patch P(x)).

[0255] FIG. 11A shows the patch 16 of FIG. 8 in the form of a local polar system of coordinates defined on the geometric domain (without explicitly showing the man shape 17) and FIG. 10B shows the same patch 16 after being mapped to a traditional polar system of coordinates. The operation of correlating the patch with a template previously described is executed among traditional polar system of coordinates like the one represented in FIG. 11B. In FIGS. 11A and 11B the interpolation weights are not shown for clarity of representation. In FIG. 12 such interpolation weights are shown as different patterns for each of the bins of the local polar system of coordinates as defined above. The patterns cover only a part of the patch for clarity reasons, in the actual method of the present invention the patterns cover all the bins of the patch.

[0256] Angular Ambiguity

[0257] If the angular coordinate of the local system of intrinsic polar coordinates is computed by ray-shooting at equispaced angles, the definition of the intrinsic convolution through the patch operator P(x) previously described suffers from angular coordinate ambiguity. In other terms, the filter can be rotated by arbitrary angle : (*g)(x)=.sub.0.sup.2.sub.0.sup.maxg(,+)(P(x))(,)dpd.

[0258] In order to deal with said angular ambiguity, the method of the present invention may consider as side information to determine the origin of the angular coordinate of the local system of intrinsic polar coordinates, one of the following: [0259] principal curvature direction at said point (only if geometric domain is a surface); [0260] direction of minimal/maximal absolute change of the data at said point;

[0261] According to another aspect of the present invention, if no side information is available, it is known that the angular ambiguity can be removed by first applying the Fourier transform with respect to the angular coordinate and then taking the absolute value, i.e. |.sub.e.sup.(P(x))(,)|. The Fourier transform translates rotational ambiguity into complex phase ambiguity, which is removed by the absolute value.

[0262] A further aspect of the method removes angular ambiguity by means of an angular max pooling procedure introduced by the present invention and comprising the steps of: [0263] creating N.sub.1 additional copies of the template g(,) by rotating it along the angular component by the angular bin magnitude, obtaining this way a total of N.sub. templates; [0264] computing the correlation .sub.0.sup.2.sub.0.sup..sup.maxg(,+) (P(x))(,)dpd of the patch with all the N.sub. rotated templates; [0265] taking the maximum operation over all the rotations, which corresponds to selecting the template with the maximum response to the given patch among the N.sub. rotated templates;

[0266] which amounts to redefine the convolution operation as (*g)(x)=max.sub.[0,2).sub.0.sup.2.sub.0.sup..sup.maxg(,+)(P(x))(,) dd.

[0267] Patch Operator Via Anisotropic Heat Kernels

[0268] According to another embodiment of the present invention, the patch operator can be defined through anisotropic heat kernels. In this setting, the main idea is to build a local representation P(x) of the given data around the point x by averaging a variety of anisotropic heat kernels at different direction. Such kernels capture local directional structures similarly to the local polar system of coordinates previously described. Varying the diffusion time of the heat kernel we can control the spread of the kernel and therefore we can encode information about the scale of the input data as well.

[0269] The construction of such local kernels requires the following steps: [0270] the computations of the thermal conductivity tensor D.sub.(x) which favors the heat diffusion towards a certain direction. The parameter controls the degree of anisotropy and is chosen depending on the data or the application considered; the computation of the rotated heat kernels D.sub.(x)=R.sub.D.sub.(x)R.sub..sup.T for various angles ; [0271] the eigendecomposition of the anisotropic LBO .sub.X=div.sub.X(D.sub.(x).sub.X(x,t)); [0272] the computation of the anisotropic heat kernels h.sub.t(x,y)=.sub.k1 e.sup.t.sup.k.sub.k(x).sub.k(y), where .sub.k, .sub.k are the anisotropic eigenvalues and eigenvectors respectively.

[0273] The thermal conductivity tensor D.sub.(x) can be defined either as

[00008]$D_{}\left(x\right)=\left(\begin{array}{cc}& 0\\ 0& 1\end{array}\right),$

where is a parameter controlling the anisotropy degree or as

[00009]$D_{}\left(x\right)=\left(\begin{array}{cc}{}_{}\left(k_M\left(x\right)\right)& 0\\ 0& {}_{}\left(k_M\left(x\right)\right)\end{array}\right),\mathrm{where}$${}_{}\left(x\right)=\frac{1}{1+.Math..Math.x.Math.}$

and k.sub.m(x), k.sub.M(x) are the minimum and maximum curvature respectively. In the latter situation, D.sub.(x) drives the diffusion in the direction of the maximum curvature k.sub.M(x).

[0274] Once the thermal conductivity tensor D.sub.(x) is defined, the anisotropic LBO is defined as .sub.X=div.sub.X(D.sub.(x).sub.X(x,t)), where D.sub.(x) is the rotated version of the operator D.sub.(x), and the anisotropic heat kernels are defined as h.sub.t(x,y)=.sub.k1e.sup.t.sup.k.sub.k(x).sub.k(y), where .sub.k, .sub.k are the anisotropic eigenvalues and eigenvectors respectively. The anisotropic patch operator is defined as

[00010]$\left(P_j\left(x\right).Math.f\right).Math.\left(t,\right)=\frac{{}_X.Math.w_j\left(x,y\right).Math.f\left(y\right).Math.\mathrm{dy},}{{}_X.Math.w_j\left(x,y\right).Math.\mathrm{dy},},j=1,\mathrm{.Math.}.Math.,J,$

where w.sub.j(x,y)=h.sub.t(x,y) and the diffusion time t plays the role of the scalar component of the local intrinsic polar coordinates. The intrinsic convolution between the input data L.sup.2(X) with a template g(t,) is thus defined as (*g)(x)=.sub.0.sup.2.sub.0.sup.t.sup.max g(t,)(P(x))(t,)dtd.

[0275] FIG. 13 schematically represents a comparison between an isotropic heat kernel 25 around the point 15 and a variety of anisotropic heat kernels 26a-28a, 26b-28b, wherein anisotropic heat kernels 26a, 27a, 28a are characterized by different orientation, while heat kernels 26a, 26b are characterized by different anisotropy intensity, and wherein the geometric domain is a mesh representing a man 17.

[0276] FIG. 14 schematically represents the same comparison of FIG. 13, where the geometric domain is a point cloud.

[0277] If the geometric domain is approximated with a triangular mesh X, the anisotropic LBO is defined as L=A.sup.1W, where A is the same as the isotropic case and

[00011]$w_{\mathrm{ij}}=\{\begin{array}{cc}\frac{1}{2}.Math.\left(\frac{{{\hat{e}}_{\mathrm{kj}},{\hat{e}}_{\mathrm{ki}}}_H}{\sin .Math..Math.{}_{\mathrm{ij}}}+\frac{{{\hat{e}}_{\mathrm{hj}},{\hat{e}}_{\mathrm{hi}}}_H}{\sin .Math..Math.{}_{\mathrm{ij}}}\right),& \mathrm{ij}E\\ -{.Math.}_{ki}.Math.w_{\mathrm{ik}},& i=j\\ 0,& \mathrm{else}\end{array}$

[0278] where .sub.kj,.sub.ki.sub.H=.sub.kj.sup.TH.sub.kj,

[00012]$H=U_{\mathrm{ijk}}\left(\begin{array}{cc}{D}_{}& 0\\ 0& 1\end{array}\right).Math.U_{\mathrm{ijk}}^T$

and U.sub.ijk=(.sub.M,.sub.m,{circumflex over (n)}) is an orthonormal reference frame attached to each triangle ijkF. The shear matrix H encodes the anisotropic scaling operated by D.sub. up to an orthogonal basis change. If D.sub.=I, .sub.kj,.sub.ki.sub.H=cos .sub.ij and the isotropic LBO discretization is obtained. FIG. 15 schematically shows a representation of the terms involved in the definition of the anisotropic LBO discretization given above. In order to allow arbitrary direction, the basis vectors U.sub.ijk is rotated on each triangle around the respective normal n by the angle , equal for all triangles. Denoting by R.sub. the corresponding 33 rotation matrix, this is equivalent to modifying the H-weighted inner product with the directed shear matrix H.sub.=R.sub.HR.sub..sup.T. The resulting weights w.sub.ij are thus obtained by using the inner products .sub.kj,.sub.ki.sub.H=.sub.ki.sup.TH.sub..sub.kj.

[0279] If the geometric domain is approximated with a point cloud instead, the procedure follows the following steps: [0280] for each point x in the point cloud, a local tangent plane is estimated by considering the points within an -ball B.sub.(x) using known techniques; [0281] the points in B.sub.(x) are projected onto the tangent plane; [0282] a Delaunay triangulation of the points projected on the tangent plane is constructed.

[0283] Once this local mesh triangulation is provided, the previous formulas still hold.

[0284] Patch Operators Via Weighting Functions on Local System of Pseudo-Coordinates

[0285] According to another aspect of the proposed invention, the patch operator may be defined as a family of weighting functions applied on a local system of pseudo-coordinates around each point on the geometric domain. Therefore, the computation of the patch operator P(x) may further comprises: [0286] the extraction of a local system of pseudo-coordinates around the point xX; [0287] the computation of weighting functions on such local coordinate system.

[0288] In such settings, the resulting intrinsic convolution layer comprises the steps of: [0289] defining a local system of multi-dimensional pseudo-coordinates u around a point xX; [0290] computing a plurality of weighting functions w.sub.i, j=1, . . . , J, acting on said pseudo-coordinates; [0291] applying said weighting functions to define a patch operator; [0292] applying the patch operator to extract a patch of the input data around a point xX; [0293] computing the correlation between said patch with a plurality of templates.

[0294] Advantageously, such definition of the patch operator is more general than the previous one and comprises, as particular instances, the definition of patch operator via Gaussians on a local system of intrinsic polar coordinates and via anisotropic heat kernels.

[0295] Local System of Pseudo-Coordinates

[0296] According to an aspect of the present invention, said local system of pseudo-coordinates u can have one or more dimensions and can be defined in different ways, comprising one or more of the following: [0297] geodesic coordinates, at each vertex yN(x) (hereinafter N(x) will indicate a neighborhood of x), is associated a 1-dimensional pseudo-coordinate u(x,y) measuring the intrinsic distance between x and y as the geodesic distance between the points, i.e. as the length of the shortest path from x to y along the geometric domain X; [0298] diffusion coordinates, measures the intrinsic distance between x and y, yN(x), in terms of a diffusion process, i.e. u(x,y)={square root over (.sub.k1e.sup.t.sup.k(.sub.k(x).sub.k(y)).sup.2)}, where .sub.k, .sub.k are the kth eigenfunction and eigenvalue of the LBO respectively; [0299] intrinsic polar coordinates, each vertex yN(x) is associated with 2-dimensional polar coordinates u(x,y)=((x,y), (x,y)), where the radial coordinate (x,y) measures the distance between the vertices x, yX in terms of one of the intrinsic distances previously introduced. The angular coordinate (x,y) can be obtained by ray-shooting from the point xX at equispaced angles. If instead, the origin of the angular coordinates of the local system of intrinsic polar coordinates is provided as a side information in terms of a direction v, (x,y) measures the angle with respect to such given direction v; [0300] vertex degree, each vertex yN(x) is associated with 1-dimensional coordinate u(x,y) measuring either the number of edges incident to y, the number of edges incident to x, or a combination thereof.

[0301] Given a point xX, the local system of pseudo-coordinates u(x,y) is computed according to one of the previous cases for every neighbor vertex yN(x) of x, where the neighborhood size can depend on the application considered. As a particular case, the size of the neighborhood N(x) of x includes the totality of the points on the geometric domain. Said operation of extracting the local system of pseudo-coordinates for the point x is repeated for all the vertices of the geometric domain X.

[0302] FIG. 16 schematically represents the radial (31) and angular (32) components of the local system of intrinsic polar coordinates centered at a point 29 of the geometric domain representing a woman 30. In particular, the radial component 31 measures the intrinsic distance of the points in the neighborhood of the central point 29. With reference to FIG. 16, the region 33 of the radial component 31 contains the closest neighbors, the region 34 contains the neighbors at a medium distance and the region 35 the farthermost points inside said neighborhood. The angular component 32 instead measures the angle between the origin of the angular coordinate 36 and a geodesic emanating form the central point 29 and reaching each neighbor point. Therefore, the angular component of 36, 37, 38, 39 corresponds to 0, 90, 180, 270 degrees respectively. The regions 40, 41, 42, 43 correspond to angular components in the range 0-90, 90-180, 180-270 and 270-360 degrees, respectively.

[0303] Weighting Functions

[0304] According to another aspect of the proposed invention, for each vertex xX several weighting functions w.sub.j(u), j=1, . . . , J, can be defined on said pseudo-coordinates u centered in each point x. Among the different weighting functions, we can consider fixed functions comprising but not limiting to: [0305] Gaussian kernels,

[00013]$w_j\left(u\right)=e^{\left(-\frac{1}{2}.Math.{\left(u-{}_j\right)}^T.Math.{.Math.}_j^{-1}.Math.\left(u-{}_j\right)\right)},$

where the mean and the covariance matrix of the kernels

[00014]$\left({}_j,{}_j=\left(\begin{array}{cc}{}_{11}^2& {}_{12}^2\\ {}_{21}^2& {}_{22}^2\end{array}\right)\right),j=1,\mathrm{.Math.}.Math.,J,$

are fixed; [0306] Anisotropic Gaussian kernels,

[00015]$w_j\left(u\right)=e^{\left(-\frac{1}{2}.Math.u^T.Math.R_{}.Math.{\stackrel{\_}{}}_j.Math.R_{}^T.Math.u\right)},$

where the covariance matrix of the kernels

[00016]${\stackrel{\_}{.Math.}}_j.Math.=\left(\begin{array}{cc}& \\ & 1\end{array}\right),j=1,\mathrm{.Math.}.Math.,J,$

and the rotation matrix

[00017]$R_{}=\left(\begin{array}{cc}\cos .Math..Math.& -\sin .Math..Math.\\ \sin .Math..Math.& \cos .Math..Math.\end{array}\right)$

are fixed; [0307] Anisotropic heat kernels; [0308] Spline kernels; [0309] Orthogonal basis functions;
parametric functions, comprising but not limiting to: [0310] Gaussian kernels,

[00018]$w_j^{}\left(u\right)=e^{\left(-\frac{1}{2}.Math.{\left(u-{}_j\right)}^T.Math.{}_j^{-1}\left(u-{}_j\right)\right)},\mathrm{where}$$=\left({}_j,{}_j=\left(\begin{array}{cc}{}_{11}^2& {}_{12}^2\\ {}_{21}^2& {}_{22}^2\end{array}\right)\right),j=1,\mathrm{.Math.}.Math.,J,$

are the parameters of the kernel; [0311] Trigonometric functions, w.sub.j.sup.)(u)=a cos(u)+b sin(u), where ={a, b, , } are the parameters associated to the weighting function;

[0312] or combinations thereof, such as sums of scaled Gaussian kernels or scaled trigonometric functions. In the following description, to uniform the notation, the fixed weighting functions w.sub.j(u) are though as parametric weighting functions w.sub.j.sup.(u) whose parameter set is empty, i.e. =.

[0313] FIG. 17 shows the level sets of three different families of weighting functions on a local system of intrinsic polar coordinates represented as a polar plot. More specifically, 45 shows the levels set of fixed Gaussian weighting functions and 49a, 50a, 51a highlight the level sets of weighting functions corresponding to a fixed angular component and small, medium and large radial component, respectively. The same visualization is repeated in 46 for the case of fixed anisotropic Gaussian weighting functions and in 47 for the case of parametric Gaussian weighting functions.

[0314] Patch Operators Via Weighting Functions on Local System of Pseudo-Coordinates

[0315] According to another embodiment of the present invention, the patch operator P(x) can be defined by applying said weighting functions w=(w.sub.1.sup.(u(x,y)), . . . , w.sub.j.sup.(u(x,y))) associated with a point xX to the input signal , i.e. P.sub.j(x)=.sub.Xw.sub.j.sup.(u(x,y)) (y) dy, j=1, . . . , J. The patch operator P(x) is computed for all the points xX of the geometric domain X.

[0316] Depending on which pseudo-coordinates u and weighting functions w.sup. are considered, different patch operators can be defined. In particular, in the following description, four different patch operators are introduced as particular instances of the proposed method by considering: [0317] Fixed Gaussians on intrinsic polar coordinates; [0318] Fixed Anisotropic Gaussians on intrinsic polar coordinates; [0319] Parametric Gaussians on intrinsic polar coordinates; [0320] Parametric Gaussians on vertex degrees;

[0321] Such framework that allows to define patch operators via weighting functions on local system of pseudo-coordinates is more general than the previous definitions of patch operators and include, as particular cases, both the patch operator via Gaussians on intrinsic polar coordinates and the patch operator via anisotropic heat kernels, previously introduced.

[0322] Patch Operator Via Fixed Gaussian Weighting Functions

[0323] In particular, the patch operator via Gaussians on intrinsic polar coordinates can be defined by considering as pseudo-coordinates said intrinsic polar coordinates and as weighting functions said fixed Gaussians.

[0324] More in details, given a point xX, such patch operator can be defined by the following three steps: [0325] construction of the local system of intrinsic polar coordinates u(x,y)=((x,y),(x,y)) on the geometric domain X, i.e. a bijection u(x,y):N(x).fwdarw.[0,.sub.max][0,2) that maps a local neighborhood N(x) of x to a system of intrinsic polar coordinates (,); [0326] construction of fixed Gaussian weighting functions

[00019]$w_j\left(,\right)=e^{\left(-\frac{1}{2}.Math.{\left(u-{}_j\right)}^T.Math.{\left(\begin{array}{cc}{}_{}^2& \\ & {}_{}^2\end{array}\right)}^{-1}.Math.\left(u-{}_j\right)\right)},$

where .sub.,.sub. a are the mean of the Gaussian along the radial and angular directions respectively. This step is agnostic on the data ; [0327] construction of the patch operator, i.e. (P.sub.j(x))(, )=.sub.Xw.sub.j.sup.(, )(y) dy, j=1, . . . , J, by applying said weights w.sub.j(, ) to said input data .

[0328] Patch Operator Via Fixed Anisotropic Gaussian Weighting Functions

[0329] The patch operator via anisotropic heat kernels can be defined by considering as pseudo-coordinates said intrinsic polar coordinates and as weighting functions fixed anisotropic Gaussians, i.e.

[00020]$w_j\left(u\right)=e^{\left(-\frac{1}{2}.Math.u^T.Math.R_{}.Math.{\stackrel{\_}{}}_j.Math.R_{}^T.Math.u\right)},$

where u(x,y)=((x,y),(x,y)) and the covariance matrix of the kernels

[00021]${\stackrel{\_}{.Math.}}_j.Math.=\left(\begin{array}{cc}& \\ & 1\end{array}\right),j=1,\mathrm{.Math.}.Math.,J,$

and the rotation matrix

[00022]$R_{}=\left(\begin{array}{cc}\cos .Math..Math.& -\sin .Math..Math.\\ \sin .Math..Math.& \cos .Math..Math.\end{array}\right)$

are fixed.

[0330] Patch Operator Via Parametric Gaussian Weighting Functions on Surfaces

[0331] According to another embodiment of the present invention, the patch operator can be defined by keeping the same local system of intrinsic polar coordinates but choosing parametric weighting functions, such as parametric Gaussians, rather than fixed weighting functions. The main idea behind the use of parametric weighting functions instead of fixed ones, is to add more flexibility to the framework by choosing different parameters able to adapt the weighting functions to each vertex rather than using the same fixed parameters over the whole geometric domain. The parameters of the parametric Gaussians are part of the parameters of the intrinsic convolution layer and are optimized during the training procedure.

[0332] More in details, the parametric Gaussian weighting functions are defined as

[00023]$w_j^{}\left(u\right)=e^{\left(-\frac{1}{2}.Math.{\left(u-{}_j\right)}^T.Math.{}_j^{-1}\left(u-{}_j\right)\right)},\mathrm{where}$$=\left({}_j,{}_j=\left(\begin{array}{cc}{}_{11}^2& {}_{12}^2\\ {}_{21}^2& {}_{22}^2\end{array}\right)\right),j=1,\mathrm{.Math.}.Math.,J,$

are the learnable parameters of the kernel. Similarly to the previous instances of the current invention, the patch operator is defined as P.sub.j(x)=.sub.Xw.sub.j.sup.)(u(x,y)) (y) dy, j=1, . . . , J, and the intrinsic convolution operation between the input data and the template g is defined as (*g) (x)=.sub.j=1.sup.Jg.sub.j(P.sub.j(x)). More complex parametric weighting functions may be chosen, including non-linear transformation of pseudo-coordinates u before passing them to the parametric Gaussian kernels.

[0333] Patch Operator Via Parametric Gaussian Weighting Functions on Undirected Graphs

[0334] Another aspect of the method of the present invention allows to define the patch operator, and consequently the intrinsic convolution operation, on graphs.

[0335] In this setting, the local system of pseudo-coordinates u can be defined in terms of the vertex degrees and the weighting functions can be defined as a family of parametric Gaussians thereon. The parameters of the parametric Gaussians are part of the parameters of the intrinsic convolution layer and are optimized during the training procedure.

[0336] More in details, given the undirected graph (V,E), where V={1, . . . , N} is the set of N vertices and E.Math.VV is the set of edges (since the graph is undirected, edges does not have a preferred direction, i.e. if (i,j)E, then (j,i)E).

[0337] In this setting, the local system of pseudo-coordinates for the node xV is defined as

[00024]$u\left(x,y\right)=A\left(\frac{1}{\sqrt{\mathrm{deg}\left(x\right).Math.}},\frac{1}{\sqrt{\mathrm{deg}\left(y\right)}}\right)+b,$

where deg(x) denotes the degree of the vertex xV (the number of edges incident to it), A is a matrix of parameters of size d2, b is a vector of parameters of size d1. The parameter d is chosen according to the application or the input data dimension (usually d=2, 3). The matrix elements a.sub.ij, i=1, . . . , d, j=1, 2, and the vector elements b.sub.i, i=1, . . . , d, are additional parameters of the intrinsic convolutional layer and are optimized during the learning procedure.

[0338] On such local pseudo-coordinates, the weighting functions are defined as the parametric Gaussians

[00025]$w_j^{}\left(u\left(x,y\right)\right)=e^{\left(\frac{1}{2}.Math.{\left(u\left(x,y\right)-{}_j\right)}^T.Math.{}_j^{-1}\left(u\left(x,y\right)-{}_j\right)\right)},$

where (.sub.j,.sub.j), j=1, . . . , J are the covariance matrices and mean vector representing the parameters of the Gaussian kernels. Analogously to the previous instances of the proposed method, the patch operator is defined as P.sub.j(x)=.sub.Xw.sub.j.sup.(u(x,y)) (y) dy, j=1, . . . , J, and the intrinsic convolution operation between the input data and the template g is defined as (*g)(x)=.sub.j=1.sup.Jg.sub.j(P.sub.j(x) ).

[0339] Patch Operator Via Parametric Weighting Functions on Raster Scans

[0340] As a particular case, a geometric domain X can be provided in terms of a raster scan, i.e. a parametric surface (x.sub.1, x.sub.2): .sup.2.fwdarw., describing the depth of the geometric domain X at the point (x.sub.1, x.sub.2). In such case, the geometric domain can be discretized as a regular grid on the parameterization domain whose vertices are associated with the depth value of the corresponding points on said geometric domain. In such settings, the intrinsic convolution layer can be applied in a sliding window fashion, in the same way known convolution layer are applied on images.

[0341] More specifically, with reference to FIG. 18, the sliding window operation on the raster scan 52 comprises the steps of: [0342] determining the central point 53a of the window 54a; [0343] extracting a block of points (dark gray region) around said central point 53a of the window 54a; [0344] defining said multi-dimensional pseudo-coordinates for each point of said block of points; [0345] computing said plurality of weighting functions acting on said multi-dimensional pseudo-coordinates; [0346] applying said weighting functions to define said patch operator extracting a local representation of the input data at the points of said block of points; [0347] computing the correlation of said patch resulting from the extraction with a plurality of templates;

[0348] and moving the window 54a to the next adjacent location 53b obtaining in this way the window 54b (light gray region). The previous steps are repeated on the window 54b.

[0349] Patch Operator Via Parametric Weighting Functions on Graph Motifs

[0350] According to another embodiment of the present invention, said intrinsic convolution layer can be defined on directed graphs as well, by first transforming them in a family of weighted undirected graphs whose weights depends on a given set of graph motifs.

[0351] A graph motif is a sub-graph which plays the role of a building block for describing more complex graphs or networks. FIG. 19 schematically shows how graph motifs M.sub.i, i=1, . . . , 13, can be defined in the simple case of a directed graph with three vertices (55, 56, 57) and three edges (58, 59, 60). Different graph motifs capture different structures and connections that may appear for instance in a social network tracking the behavior of a group of people.

[0352] Given a directed graph G and a graph motif M.sub.i, a weighted undirected graph {tilde over (G)}.sub.i is defined by considering as vertices the same vertices of G and by defining the edge weights as the occurrence of the graph motif M.sub.i. Once the weighted undirected graph {tilde over (G)}.sub.i is extracted from G, the patch operator on {tilde over (G)}.sub.i is extracted by defining a local system of pseudo-coordinates and a family of weighting functions thereon. As a particular example, the patch operator can be defined following the same construction of the patch operator via parametric Gaussian weighting functions on undirected graphs, i.e. by considering as pseudo-coordinates non-linear functions of the vertex degree and as weighting function a family of parametric Gaussians thereon. Other examples of pseudo-coordinates include but are not limited to geodesic or diffusion distance from a vertex.

[0353] Other Layers

[0354] Following the same construction of known convolutional neural networks, the proposed method combines other known layers with the intrinsic convolution layer either to achieve better performance or to adapt the presented method to the desired application.

[0355] In particular, the proposed method consists of several layers than can be applied subsequently, by this meaning that the output of the previous layer is used as input into the subsequent one to form a hierarchy of transformations. Other known layers include: [0356] linear layer or fully connected layer, given an I-dimensional input f.sup.in=(.sub.1.sup.in(x), . . . , .sub.I.sup.in(x)) produces a Q-dimensional output f.sup.out=(.sub.1.sup.out(x), . . . , .sub.Q.sup.out(x)) as linear combination of the input channels with a set of learnable parameters w, .sub.q.sup.out(x)=.sub.i=1.sup.Iw.sub.qi.sub.p.sup.in, q=1, . . . , Q; [0357] non-linear layer, optionally it follows a linear layer by applying a non-linear function to the output of the linear layer, i.e. .sub.q.sup.out(x)=(.sub.i.sup.in), i=1, . . . , Q. The non-linear functions are often referred to as activation functions. Most common activation functions include, but are not limited to: [0358] ReLU, (.sub.i.sup.in)=max{0, .sub.i.sup.in}; [0359] logististic,

[00026]$\left(f_i^{i.Math..Math.n}\right)=\frac{1}{1+e^{-f_i^{i.Math..Math.n}}};$ [0360] hyperpolic tangent, (.sub.i.sup.in)=tan h(.sub.i.sup.in); [0361] softmax,

[00027]$\left(f_i^{i.Math..Math.n}\right)=\frac{e^{f_i^{i.Math..Math.n}}}{\underset{p=1}{\overset{P}{.Math.}}.Math.e^{f_{i.Math.}^{i.Math..Math.n}}};$ [0362] spatial pooling layer, consists in a spatial aggregation of input data to enforce a hierarchy structure in the learned features. More specifically, the spatial pooling layer comprises the steps of: [0363] determining a subset of points x.sub.1, . . . , x.sub.N on the geometric domain X; [0364] for each point x.sub.i of the subset, determining the neighbors x.sub.i.sup.1, . . . , x.sub.i.sup.M, on the geometric domain; [0365] computing an averaging operation on input data over the neighbors for all the points on said subset.

[0366] Averaging operations include the known [0367] arithmetic mean,

[00028]$f\left(x_i\right)=\frac{1}{M}.Math.\underset{j=1}{\overset{M}{.Math.}}.Math.f\left(x_i^j\right);$ [0368] a maximum operation, (x.sub.i)=max.sub.j=1, . . . , M(x.sub.i.sup.j); and the novel [0369] weighted average with weights depending on an intrinsic distance between points, (x.sub.i)=.sub.j=1.sup.Md.sub.X(x.sub.i, x.sub.i.sup.j)(x.sub.i,x.sub.i.sup.j), where d.sub.X(.Math., .Math.) is one of the intrinsic distances previously mentioned; [0370] weighted average with weights depending on local volume elements (x.sub.i)=.sub.j=1.sup.Mvol.sub.X(x.sub.i.sup.j)(x.sub.i.sup.j), where vol.sub.X(x.sub.i.sup.j) can be the area of a local patch around the point x.sub.i.sup.j; [0371] covariance layer, it is used in applications such as retrieval where one needs to aggregate the point-wise descriptors to produce a global shape descriptor f.sub.out(x)=.sub.X(f.sup.in(x))(f.sup.in(x)).sup.Tdx, where f.sup.in(x)=(.sub.1.sup.in(x), . . . , .sub.I.sup.in(x)) is a P-dimensional input vector, =.sub.X f.sup.in(x) dx, and f.sup.out(x) is a II matrix column-stacked into a I.sup.2-dimensional vector. Typically, it is considered just before the output layer.

[0372] Each of the previous layers has input data and output data and the output data of one layer in the sequence can be given as input to any subsequent layer, or layers, in the sequence. The present invention does not impose any constraint on the composition of said layers, including any limit on the order or the number of layers to consider.

[0373] Such sequence of customized layers can be thought of as a non-linear hierarchical parametric function .sub.(F), where F=(f(x.sub.1), . . . , f(x.sub.N)) is a IN matrix of input features at all the points of the geometric domain, and denotes the parameters of all the layers, and .sub.(F)=.sub.N.sub.N-1 . . . .sub.0(F), where .sub.i indicates one of the aforementioned layers.

[0374] Cost Functions

[0375] Advantageously, depending on the application in mind, the present invention allows to learn these parameters by minimizing some task specific cost function. The present invention allows to learn the parameters , i.e. intrinsic convolution templates, B-spline interpolation weights, linear combination weights, etc.

[0376] The applications that the proposed invention can deal with, include: [0377] invariant descriptors, i.e. finding descriptors that are the most similar as possible for corresponding points, or positives, and the most dissimilar as possible at non-corresponding points, or negatives; [0378] correspondences, i.e. labelling each vertex of a query geometric domain X with the index of a corresponding point on some reference geometric domain Y. [0379] shape retrieval, i.e. producing a global shape descriptor that discriminates between shape classes.

[0380] For this purpose, the present invention consider a siamese network configuration, composed of two identical copies of the same intrinsic convolutional neural network model sharing the same parameterization and fed by pairs of knowingly similar or dissimilar samples, and minimize a cost function such as, l()=(1).sub.i=1.sup.|T.sup.+.sup.|.sub.(f.sub.i).sub.(f.sub.i.sup.+).sup.2+.sub.i=1.sup.|T.sup..sup.|(.sub.(f.sub.i).sub.(f.sub.i.sup.)).sub.+.sup.2, where [0,1] is a parameter trading off between the positive and the negative losses, is a margin, (x).sub.+=max{0, x} and T.sub.={(f.sub.i, f.sub.i.sup.+)} denotes the sets of positive and negative pairs, respectively. Variants of such cost function can be used as well.

[0381] Other examples of cost functions do not rely on the siamese network construction, therefore they can be minimized by only one intrinsic convolutional neural network. Example of such cost functions are: [0382] multinomial regression loss, l()=.sub.i=1.sup.|T|e.sub.ji log .sub.(f.sub.i), where e.sub.i is a unit vector with one at index i, and T={f(x.sub.i),j.sub.i} represents a set of known correspondences. [0383] regression loss, l()=.sub.i=1.sup.|T|.sub.(f.sub.i)j.sub.i.sub.p where j.sub.i represents the desired output of the model (teacher signal) and .sub.p denotes the p-norm (usually p=2). As means of example j.sub.i can be the spectral coefficients of some target functions.

[0384] In some embodiments, the methods and processes described herein can be embodied as code and/or data. The software code and data described herein can be stored on one or more machine-readable media (e.g., computer-readable media), which may include any device or medium that can store code and/or data for use by a computer system. When a computer system reads and executes the code and/or data stored on a computer-readable medium, the computer system performs the methods and processes embodied as data structures and code stored within the computer-readable storage medium.

[0385] It should be appreciated by those skilled in the art that machine-readable media (e.g., computer-readable media) include removable and non-removable structures/devices that can be used for storage of information, such as computer-readable instructions, data structures, program modules, and other data used by a computing system/environment. A computer-readable medium includes, but is not limited to, volatile memory such as random access memories (RAM, DRAM, SRAM); and non-volatile memory such as flash memory, various read-only-memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic/ferroelectric memories (MRAM, FeRAM), and magnetic and optical storage devices (hard drives, magnetic tape, CDs, DVDs); network devices; or other media now known or later developed that is capable of storing computer-readable information/data. Computer-readable media should not be construed or interpreted to include any propagating signals. A computer-readable medium that can be used with embodiments of the subject invention can be, for example, a compact disc (CD), digital video disc (DVD), flash memory device, volatile memory, or a hard disk drive (HDD), such as an external HDD or the HDD of a computing device, though embodiments are not limited thereto. A computing device can be, for example, a laptop computer, desktop computer, server, cell phone, or tablet, though embodiments are not limited thereto.

[0386] In some embodiments, one or more (or all) of the steps performed in any of the methods of the subject invention can be performed by one or more processors (e.g., one or more computer processors). For example, any or all of the means for applying an intrinsic convolution layer, the means for applying one or more layers as described herein, the means for determining similarity between two geometric objects, and the means for determining correspondence between objects from a class of geometric objects and a reference object can include or be a processor (e.g., a computer processor) or other computing device.

[0387] It should be understood that the examples and embodiments described herein are for illustrative purposes only and that various modifications or changes in light thereof will be suggested to persons skilled in the art and are to be included within the spirit and purview of this application.

[0388] All patents, patent applications, provisional applications, and publications referred to or cited herein are incorporated by reference in their entirety, including all figures and tables, to the extent they are not inconsistent with the explicit teachings of this specification.