Method for Obtaining a Spatial Pattern of an Anatomical Structure of a Subject, Related System and Markers
20220414934 · 2022-12-29
Assignee
Inventors
Cpc classification
A61B5/0077
HUMAN NECESSITIES
G06T7/80
PHYSICS
G16H50/20
PHYSICS
A61B5/706
HUMAN NECESSITIES
A61B5/4561
HUMAN NECESSITIES
International classification
G06T7/80
PHYSICS
A61B5/00
HUMAN NECESSITIES
A61B5/107
HUMAN NECESSITIES
Abstract
A method for obtaining a spatial pattern of an anatomical structure of a subject includes comprising the steps of a) acquiring, from at least one digital image capturing device, at least one uncalibrated image of a calibration reference applied on a surface configured to receive the subject, the calibration reference having at least one known dimension and defining at least one known direction; b) defining an absolute calibrated reference system of three coordinates based on the calibration reference depicted in the at least one uncalibrated image; and c) acquiring, from the at least one digital image capturing device, at least a first and at least a second calibrated image of a plurality of markers applied on a corresponding plurality of body landmarks of the anatomical structure of the subject at respective contact points with the body landmarks, the plurality of markers being arranged within the absolute calibrated reference system,
Claims
1-52. (canceled)
53. A method for obtaining a spatial pattern of an anatomical structure of a subject, comprising the steps of: a) acquiring, from at least one digital image capturing device, at least one uncalibrated image of a calibration reference applied on a surface configured to receive the subject, said calibration reference having at least one known dimension and defining at least one known direction; b) defining an absolute calibrated reference system of three coordinates based on the calibration reference depicted in said at least one uncalibrated image; and c) acquiring, from said at least one digital image capturing device, at least a first and at least a second calibrated image of a plurality of markers applied on a corresponding plurality of body landmarks of the anatomical structure of the subject at respective contact points with the body landmarks, said plurality of markers being arranged within said absolute calibrated reference system, wherein said at least a first calibrated image depicts a marker spatial arrangement in a first plane and said at least a second calibrated image depicts the marker spatial arrangement in a second plane different from the first plane, the first and second planes being representative of a relative orientation of the same plurality of markers with respect to the at least one digital image capturing device, wherein said markers of said plurality of markers each comprise a respective three-dimensional main body shaped as a polyhedron of N faces and having N−1 exposed faces arranged in a known geometric relationship with the contact point of the marker with the respective body landmark, and wherein at least one of said N−1 exposed faces of each marker is identifiable in said at least first and second calibrated images by means of an image recognition algorithm.
54. The method according to claim 53, further comprising the step of: d) determining for each marker of said plurality of markers a position of the contact point with the respective body landmark of the subject within said absolute calibrated reference system.
55. The method according to claim 53, wherein said steps a)-d) are executed by a same processor.
56. The method according to claim 53, wherein said steps a)-c) are executed by a first processor, and wherein said step d) is executed by a second processor, said second processor being a remote processor with respect to said first processor.
57. The method according to claim 53, wherein the first plane and the second plane are perpendicular to each other.
58. The method according to claim 57, wherein said first plane of the first calibrated image substantially coincides with a coronal plane of the subject in standing position on the surface, and said second plane of the second calibrated image substantially coincides with a sagittal plane of the subject in standing position on the surface.
59. The method according to claim 53, wherein said first and second planes share a common vertical coordinate.
60. The method according to claim 53, wherein the main body of each marker of said plurality of markers is shaped as a regular polyhedron selected from an octahedron, a dodecahedron, an icosahedron.
61. The method according to claim 53, wherein said step d) comprises: d1) determining for each marker of the plurality of markers a first initial position of the respective contact point of the marker in the first calibrated image acquired in step c); d2) determining for each marker of the plurality of markers a second initial position of the respective contact point of the marker in the second calibrated image acquired in step c); and d3) combining the first and second initial positions, respectively determined in steps d1) and d2) from said first and second calibrated images, to obtain the position of each marker.
62. The method according to claim 61, wherein said steps d1) and d2) each comprise, for each marker of the plurality of markers depicted in said at least a first, respectively said at least a second, calibrated image: d4) identifying one or more exposed faces of the marker by recognizing a respective distinctive feature thereof; d5) determining in said at least a first calibrated image, respectively, in said at least a second calibrated image, based on an orientation of the one or more exposed faces of the marker recognized, a position of a center of symmetry of the main body of the marker; d6) determining in said at least a first calibrated image, respectively, in said at least a second calibrated image, based on an orientation of the one or more exposed faces of the marker recognized, the direction of an axis of symmetry of the marker passing through a contact surface of the marker with the respective body landmark; and d7) determining in said at least a first calibrated image, respectively, in said at least a second calibrated image, the first initial position, respectively, the second initial position, of the contact point of the marker along said axis of symmetry of the marker, at a known distance from the center of symmetry of the main body of the marker.
63. The method according to claim 62, wherein said step d) comprises defining a relative reference system of coordinates for each distinctive feature recognized, said relative reference system of coordinates being centered at a center of symmetry of the distinctive feature recognized.
64. The method according to claim 61, wherein said step d3) comprises averaging a common axial coordinate between said first and said second initial positions respectively determined from said at least a first and said at least a second calibrated images.
65. The method according to claim 53, further comprising the step of: e) fitting the positions of the contact points of markers of said plurality of markers with a curve.
66. The method according claim 65, wherein said step e) of fitting the positions of the contact points of the markers with a curve is carried out by means of an interpolation operation selected from polynomial interpolation and spline interpolation.
67. The method according to claim 65, further comprising the step of: f) calculating, from the curve fitted in step e), parameters of interest related to the anatomical structure of the skeleton of the subject, and/or displaying said curve.
68. The method according to claim 67, wherein said parameters of interest calculated in step f) are selected among distances, areas, angles, curvatures.
69. The method according to claim 67, wherein said parameters of interest calculated in step f) comprise a Cobb angle.
70. The method according to claim 53, wherein said calibration reference comprises at least three tags applied onto said surface, a first and a second tag being positioned at a first known distance from one another along a first axis, and said second tag and a third tag being positioned at a second known distance from one another along a second axis perpendicular to said first axis, and wherein said first axis and said second axis are parallel, when said at least a first and said at least a second calibrated image are captured by the digital image capturing device, to a latero-lateral direction and, respectively, to a postero-anterior direction of the subject.
71. The method according to claim 53, wherein said surface has a first reference plane substantially coinciding, in use, with a coronal plane of the subject standing thereon, and a second reference plane substantially coinciding, in use, with a sagittal plane of the subject standing thereon.
72. The method according to claim 71, wherein said first plane of the first calibrated image is substantially parallel to or coincides with the first reference plane of the surface, and said second plane of the second calibrated image is substantially parallel to or coincides with the second reference plane of the surface.
73. The method according to claim 71, wherein said at least one uncalibrated image acquired in step a) is captured by the at least one digital image capturing device with a focal axis thereof substantially aligned to either the first reference plane or the second reference plane of said surface.
74. The method according to claim 71, wherein said at least a first and said at least a second calibrated image acquired in step c) are captured by the at least one digital image capturing device with the focal axis thereof substantially aligned to the second reference plane, respectively to the first reference plane, of said surface.
75. The method according to claim 71, wherein said at least one uncalibrated image acquired in step a), and said at least a first and said at least a second calibrated image acquired in step c), are captured by the at least one digital image capturing device with a lens thereof positioned at a same fixed distance along the focal axis from either the first reference plane or the second reference plane of said surface.
76. The method according to claim 53, wherein said surface configured to receive the subject is an upper surface of a support rotatably mounted onto a stationary stand configured to be laid on the ground, said support being preferably rotatable between fixed angular positions angularly spaced of 90°.
77. The method according to claim 76, wherein said support comprises a guide configured to set up an angle between feet of the subject positioned thereon.
78. The method according to claim 76, wherein said first and second reference planes of the surface are median planes of the support.
79. The method according to claim 53, wherein said at least one uncalibrated image acquired in step a), and said at least a first and said at least a second calibrated image acquired in step c), are captured by the at least one digital image capturing device with the focal axis thereof positioned at a same vertical distance from the surface.
80. The method according to claim 53, wherein said at least one uncalibrated image acquired in step a), and said at least a first and said at least a second calibrated image acquired in step c), are captured by the at least one digital image capturing device with the focal axis thereof substantially parallel to said surface.
81. The method according to claim 53, wherein said at least a first and said at least a second calibrated image acquired in step c) are captured by the at least one digital image capturing device with the focal axis thereof substantially parallel to the postero-anterior direction, respectively to the latero-lateral direction of the subject.
82. The method according to claim 53, wherein said at least one uncalibrated image acquired in step a) is captured by the at least one digital image capturing device with the optic axis thereof substantially parallel either to the postero-anterior direction or to the latero-lateral direction of the subject.
83. The method according to claim 53, wherein said step c) further comprises acquiring a third calibrated image of the marker spatial arrangement on either the first or second plane, viewed from an opposite direction with respect to said at least a first or said at least a second calibrated image, respectively, wherein said step d) further comprises: d21) determining for each marker of the plurality of markers a third initial position of the respective contact point of the marker in the third calibrated image acquired in step c); wherein said step d21) comprises, for each marker of the plurality of markers depicted in said third calibrated image: d41) identifying one or more exposed faces of the marker by recognizing a respective distinctive feature thereof; d51) determining in the third calibrated image, based on an orientation of the one or more exposed faces of the marker recognized, a position of a center of symmetry of the main body of the marker; d61) determining in the third calibrated image, based on an orientation of the one or more exposed faces of the marker recognized, the direction of an axis of symmetry of the marker passing through a contact surface of the marker with the respective body landmark; and d71) determining in the third calibrated image, the third initial position of the contact point of the marker along said axis of symmetry of the marker, at a known distance from the center of symmetry of the main body of the marker; wherein step d3) comprises combining the first, the second and the third initial positions, respectively determined in steps d1), d2) and d21) from said at least a first, said at least a second and said third calibrated images, to obtain the position of each marker; and wherein said step d3) comprises averaging a first common axial coordinate between said first, second and third initial positions respectively determined from said at least a first, said at least a second and said third calibrated images, and averaging a second common axial coordinate between said second and third initial positions.
84. The method according to claim 53, wherein said plurality of markers comprises at least four markers, and said plurality of body landmarks comprises at least: a first body landmark selected among cervical vertebrae C4 and C5; a second body landmark selected among dorsal vertebrae D4, D5, D6 and D7; a third body landmark selected among dorsal vertebrae D8, D9, D10, D11 and D12; a fourth body landmark selected among lumbar vertebrae L1, L2, L3, L4 and L5.
85. The method according to claim 84, wherein said plurality of markers comprises at least six markers, and said plurality of body landmarks further comprises the left and right acromions.
86. The method according to claim 53, wherein said anatomical structure of the subject is the spine.
87. A system for obtaining a spatial pattern of an anatomical structure of a subject, said system comprising: a calibration reference applied on a surface configured to receive the subject, said calibration reference having at least one known dimension and defining at least one known direction; a plurality of three-dimensional markers configured to be applied on a corresponding plurality of body landmarks of the anatomical structure of the subject at respective contact points, each marker comprising a respective three-dimensional main body, shaped as a polyhedron of N faces, and having N−1 exposed faces arranged in a known geometric relationship with the contact point of the marker with the respective body landmark, wherein at least one of said N−1 exposed faces of each marker is identifiable, by means of an image recognition algorithm, in an image of the marker captured by a digital image capturing device; and at least one digital image capturing device configured to: (i) capture at least one uncalibrated image of said calibration reference; and (ii) capture at least a first and at least a second calibrated image of said plurality of markers applied on said corresponding plurality of body landmarks and arranged within said absolute calibrated reference system defined based on said at least one uncalibrated image of the calibration reference, wherein said at least a first calibrated image depicts a marker spatial arrangement in a first plane and said at least a second calibrated image depicts the marker spatial arrangement in a second plane different from the first plane, the first and second planes being representative of a relative orientation of the plurality of markers with respect to the at least one digital image capturing device.
88. The method according to claim 87, further comprising: a processor programmed to: (i) acquire said at least one uncalibrated image of the calibration reference from the at least one digital image capturing device; (ii) define said absolute calibrated reference system of coordinates based on said at least one uncalibrated image; (iii) acquire said at least a first and at least a second calibrated images from the at least one digital image capturing device; and (iv) determine for each marker of said plurality of markers a position of the contact point with the respective body landmark of the subject within said absolute calibrated reference system.
89. The method according to claim 87, further comprising: a first processor programmed to: (i) acquire said at least one uncalibrated image of the calibration reference from the at least one digital image capturing device; (ii) define said absolute calibrated reference system of coordinates based on said at least one uncalibrated image; (iii) acquire said at least a first and said at least a second calibrated images from the at least one digital image capturing device; and (iii_bis) transmit said at least a first and at least a second calibrated images of said plurality of markers to a second processor; and the second processor programmed to: (iii_ter) acquire said at least a first and said at least a second calibrated images of said plurality of markers from said first processor; and (iv) determine for each marker of said plurality of markers the position of the contact point with the respective body landmark of the subject within said absolute calibrated reference system, said second processor being remote with respect to said first processor.
90. The method according to claim 87, wherein the main body of each marker of said plurality of markers is shaped as a regular polyhedron.
91. The method according to claim 87, wherein each marker of said plurality of markers is provided with a base having a surface configured to contact the respective body landmark.
92. The method according to claim 87, further comprising: a support having said calibration reference positioned on an upper surface thereof, said support being configured to receive the subject in standing position.
93. The method according to claim 92, wherein said support is rotatably mounted onto a stationary stand configured to be laid on the ground.
94. The method according to claim 87, wherein said support comprises a guide configured to set up an angle between feet of the subject positioned thereon.
95. The method according to claim 87, wherein said surface has a first reference plane substantially coinciding, in use, with a coronal plane of the subject standing thereon, and a second reference plane substantially coinciding, in use, with a sagittal plane of the subject standing thereon.
96. The method according to claim 95, wherein said first and second reference planes of the surface are median planes of the support.
97. A marker configured to obtain a spatial pattern of an anatomical structure of a subject, said marker being configured to be applied on a body landmark of the anatomical structure of the subject at a respective contact point and comprising a three-dimensional main body shaped as a polyhedron of N faces, having N−1 exposed faces arranged in a known geometric relationship with the contact point of the marker with the respective body landmark; wherein at least one of said N−1 exposed faces of the marker is identifiable, by means of an image recognition algorithm, in an image of the marker captured by a digital image capturing device.
98. The marker according to claim 97, further comprising a base having a surface configured to contact the respective landmark.
99. The marker according to claim 98, wherein said base comprises an elongated stem that projects from a proximal face of the marker, and a plate on which the contact surface of the marker is defined.
100. The marker according to claim 97, wherein each exposed face is provided with a distinctive feature.
101. The marker according to claim 97, wherein one or more groups of exposed faces of the marker each share a same distinctive feature.
102. The marker according to claim 97, wherein said distinctive feature comprises graphic sign over a background of a contrasting colour.
103. The marker according to claim 102, wherein the main body is shaped as a regular polyhedron.
104. The marker according to claim 97, wherein the main body has a shape selected from an octahedron, a dodecahedron, an icosahedron.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0223] Additional features and advantages of the present invention will be better illustrated by the following detailed description of some of preferred embodiments thereof, made with reference to the accompanying drawings, in which structural or functional elements having the same or similar function are indicated by identical or similar reference numbers.
[0224] In the drawings:
[0225]
[0226]
[0227]
[0228]
[0229]
[0230]
[0231]
[0232]
[0233]
[0234]
[0235]
[0236]
[0237]
[0238]
DETAILED DESCRIPTION OF CURRENTLY PREFERRED EMBODIMENTS
[0239] Throughout the various embodiments disclosed herein, similar elements or elements having a similar function are indicated by the same reference numbers.
[0240] In the following description, certain specific details are set forth in order to provide a thorough understanding of various disclosed embodiments. However, one skilled in the relevant art will recognize that embodiments may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with the various embodiments have not been shown or described in detail to avoid unnecessarily obscuring descriptions of the embodiments.
[0241] With reference to
[0242] The marker 10 illustrated in
[0243] The main body 12 is preferably shaped as a dodecahedron, as illustrated, and thus comprises twelve faces shaped as regular pentagons. In alternative embodiments not shown, the main body 12 can be shaped as a different regular polyhedron, such as for example an octahedron, an icosahedron, etc.
[0244] The base 14 comprises a contact surface 16 configured to contact and be attached to the skin of a human or animal subject at specific locations, corresponding to predefined body landmarks of an anatomical structure of the skeleton of the subject of interest.
[0245] In the case illustrated, the base 14 is in particular a disc-shaped plate stably connected to a proximal face 15 of the main body 12, and the contact surface 16 is thus substantially circular.
[0246] The base 14 can be integral with the main body 12 of the marker 10, or it can be a distinct element stably coupled therewith, releasably or unreleasably, by providing suitable connecting elements (not visible) at the interface with the main body 12 of the marker 10. By way of example, the base 14 can be coupled to the main body 12 by means of gluing, interference fit or other types of coupling.
[0247] Markers 10 are preferably attached to the skin of the subject, through the base 14, by means of a suitable hypoallergenic adhesive (not shown). By way of a non limiting example, a double sided adhesive sheet made of a polymeric material can be employed. Such adhesive sheet can be advantageously pre-attached to the contact surface 16, equipped with a release sheet to be peeled off before applying the marker 10 onto the subject.
[0248] An example of a suitable adhesive sheet is a 20 mm circular adhesive pad provided by 3M.
[0249] In the present disclosure, the other faces of the main body 12, except the proximal face 15 connected to the base 14, are all referred to as exposed faces 18.
[0250] Each exposed face 18 has a respective distinctive feature 20a, 20b, 20c provided thereon. Distinctive features 20a, 20b, 20c are in particular glyphs (referred to using the same reference numbers 20a, 20b, 20c), namely simple graphic elements that are recognizable by processing the image according to a suitable glyph recognition algorithm. Such glyphs can be for example printed on the faces of the main body of the markers.
[0251] In the preferred embodiments illustrated, a distal face 24, opposite to the proximal face 15 of the polyhedral main body 12 of the marker 10, has a first glyph referred to as distal glyph 20a; the five faces immediately adjacent to the distal face 24, referred to as first lateral faces 26, have a second common glyph, referred to as first lateral glyph 20b; and the five faces immediately adjacent to the base 14, referred to as second lateral faces 28, have a third common glyph, referred to as second lateral glyph 20c.
[0252] Glyphs 20a, 20b, 20c, better illustrated in
[0253] Glyphs 20a-20f are obtained by suitably distributing dark squared modules within a grid (e.g. grid 30 shown in
[0254] By way of example, the grid 30 shown in
[0255] The provision of a marker 10 having a three-dimensional main body 12, projecting at a certain distance from the skin of the subject when the marker is applied thereon, allows the marker 10 to be at least partially visible also when it is applied in body regions of the subject that might be subject to visual obstruction—when viewed from certain angles—due to the presence of protruding body parts, such as the scapulae.
[0256]
[0257] The stem 32 is preferably shaped as a hollow cylinder, and can comprise, as illustrated, one or more openings 38 that lighten the overall marker 40. The length of the stem 32 can be tailored based on a desired projecting distance of the marker 40 with respect to the skin of the subject, so that the marker 40 may be suitable for use also in regions of the body of the subject that are particularly subject to visual obstruction.
[0258] It should be noted that although markers 10, 40 have been described as comprising a distinct base 14, 44 on which the contact surface 16, 46 of the markers with the skin of the subject is defined, in simpler embodiments (not shown) the contact surface can be embodied by the proximal face 15 itself of the main body 12.
[0259] The markers 10, 40 can be manufactured from any hypoallergenic light, rigid material, and both the main body 12 and the stem 32, where provided, can be solid or hollow. Light plastic materials are particularly suitable for manufacturing the markers.
[0260] Preferably, the markers 10, 40 are made of a polymeric material.
[0261] More preferably, the polymeric material is selected from poly-lactic acid (PLA), acrylonitrile-butadiene-styrene (ABS), high impact polystyrene (HIPS), nylon, carbon fiber, polycarbonate and acrylic styrene acrylonitrile (ASA).
[0262] Preferably, in such a case the markers 10, 40 can advantageously be manufactured using 3D printing processes.
[0263] Different marker sizes, along with glyphs dimensions, can be provided.
[0264] By way of example, the main body of the markers 10, 40 may have an edge of about 12 mm and can be inscribed in a sphere having a diameter of about 33 mm.
[0265] By way of example, the contact surface 16, 46 of the markers 10, 40 may be circular and may have a diameter of about 21 mm.
[0266] By way of example, a distance between the proximal face 15 and the contact surface 16 of the marker 10 is of about 6 mm.
[0267] By way of example, a distance between the proximal face 15 and the contact surface 46 of the marker 40 is of about 20 mm.
[0268] The markers 10, 40 are employed in the method of the invention, as later described in detail with reference to
[0269] The number of markers 10, 40 employed and the selection of body landmarks depends on the complexity of the anatomical structure under analysis and on the desired biometric parameters to be subsequently estimated.
[0270] For example, for dimensional analysis of simple “linear” structures such as long bones (e.g. femur, tibia), two markers, positioned at the bone ends, are sufficient. For example, for leg length analysis, relevant landmarks are the trochanter and the outer malleolus.
[0271] For more complex structures, such as the spine, more than two markers are generally needed.
[0272]
[0273] The markers may be markers according to any embodiment of markers 10, 40 described above with reference to
[0274] In
[0275] In a different embodiment, the markers 40 comprising the stem 32 (see
[0276] As shown in
[0277] In particular, a first body landmark 50a is selected among cervical vertebrae C4 and C5 (approximately within region 52a); a second body landmark 50b is selected among dorsal vertebrae D4, D5, D6 and D7 (approximately within region 52b); a third body landmark 50c is selected among dorsal vertebrae D8, D9, D10, D11 and D12 (approximately within region 52c); and a fourth body landmark 50d is selected among lumbar vertebrae L1, L2, L3, L4 and L5 (approximately within region 52d).
[0278] Additional markers 10e, 10f can be optionally provided to improve the analysis by correlating the morphology and position of the spine S with the surrounding bone structures of the torso. For example, as illustrated, additional markers 10e, 10f are provided respectively positioned at body landmarks 50e, 50f, corresponding to the left and right acromions.
[0279] In an even more complex scenario, for analyzing the overall skeleton, (“total body” analysis), up to twenty markers may be needed, according to another embodiment (not shown) of the invention.
[0280] Besides the six markers 10a-10f illustrated in
[0281]
[0282] The system 100 preferably comprises a plurality 110 of markers (which may be the markers 10, the markers 40 or any suitable combination thereof), a digital image capturing device 112, in particular a camera, a computational device 113 including a processor 114, and a support 116.
[0283] In the preferred embodiment illustrated in
[0284] In particular, the plurality 110 of markers illustrated includes the previously described markers 40a-40d, applied on the back B of the subject H and distributed along the spine S, in a same manner as previously illustrated in
[0285] The system 100 illustrated in
[0286] Preferably, the support 116 is adapted to receive the subject H, having the markers 40a-40d attached on the spine at selected body landmarks, in a standardized standing position. The camera 112 is configured and positioned to capture images of the plurality 110 of markers, applied onto the subject H, from at least two different angles.
[0287] Images captured by the camera 112 are transferred to the processor 114 of the computational device 113 for image processing and subsequent elaboration in order to reconstruct a pattern of points of the spine S of the subject H and further determine related biometric parameters.
[0288] Preferably, the support 116 is rotatably mounted onto a stationary base (underneath the support 116 and not visible) that lays on the ground, so that the support 116 is rotatable with respect to the stationary base.
[0289] Preferably, the support 116 is rotatable between fixed angular positions angularly spaced at 90° from each other.
[0290] Preferably, the support 116 is a squared platform with a substantially flat upper surface 118.
[0291] With further reference to
[0292] Conveniently, the tags 120a, 120b, 120c, 120d constitute a calibration reference CRef used during a calibration operation of the method of the invention, as later described.
[0293] The tags 120a, 120b, 120c, 120d of the calibration reference CRef are positioned at the four corners of the upper surface 118 of the support 116. Therefore, each tag is equally spaced from adjacent ones of a distance d.sub.1=d.sub.2, and couples of tags 120a, 120b; 120b, 120c; 120c, 120d; and 120d, 120a lie along reciprocally perpendicular directions (axes x and y), respectively corresponding to a latero-lateral direction and to an postero-anterior direction of the subject H standing on the support 116. Moreover, the perpendicular directions x, y are respectively parallel to median planes t.sub.1 and t.sub.2 of the support 116.
[0294] Preferably, the support 116 further comprises, fixed on its upper surface 118, a guide 122 (illustrated only in
[0295] The oblique sides 124, 126 of the guide 122 are configured to abut, in use, to inner sides of the feet of the subject (not shown) in order to fix and “standardize” the position of the subject. The angle α defined by the sides 124, 126 is of about 30°, which is considered an optimum angle to evaluate postural issues.
[0296] In a different preferred embodiment shown in
[0297] Going back to
[0298] The camera 112 is positioned so that its focal axis F is substantially parallel to the upper surface 118 of the support 116, namely so that an angle between the focal axis F and the upper surface 118 of the support 116 is comprised between 0° and 1°.
[0299] The focal axis F is also substantially aligned with either median plane t.sub.2 (image capturing configuration of
[0300] Preferably, the camera 112 is so positioned that its lens 127 is at a predefined distance D with respect to median plane t.sub.1 (configuration of
[0301] Moreover, based on the height of the subject H, the camera 112 is positioned so that the focal axis F is at a vertical distance h from the upper surface of the support 116, said vertical distance h preferably substantially corresponding to the height of the subject's iliac crest.
[0302] In this way, the image distortion is advantageously minimized.
[0303] In the image capturing configuration of
[0304] Preferably, this first plane (x, z) corresponds to a coronal plane of the subject H carrying the markers 40a-40d.
[0305] In other words, in the image capturing configuration of
[0306] In the image capturing configuration of
[0307] Preferably, this second plane (y, z) corresponds to a sagittal plane of the subject H carrying the markers 40a-40d.
[0308] In other words, in the image capturing configuration of
[0309] By way of example, the camera is a Canon 5Ds with 50 megapixel sensor and a 18-55 lens.
[0310] The images captured by the camera 112 are then acquired by the processor 114 of the computational device 113, in order to be processed according to a preferred method according to the invention.
[0311] The processor 114 can be in data communication with the camera 112 through a suitable communication link 128, such as a cable or a wireless link.
[0312] Alternatively, data transfer between the camera 112 and the processor 114 can take place off-line, by means of movable storage devices such as USB flash drives or flash memory cards to be placed in suitable ports/readers provided in the computational device 113 and in the camera 112.
[0313] The processor 114, which is for example a central processor (CPU) of the computational device 113, may be coupled to a random access memory (RAM), preferably of at least 8 GB, and to a read-only memory (ROM). The ROM may also be other types of storage media to store programs, such as programmable ROM (PROM), erasable PROM (EPROM), etc.
[0314] The computational device 113 may include a hard drive, preferably of at least 256 GB.
[0315] The processor 114 may communicate with other internal and/or external components through input/output (I/O) circuitry and bussing to provide control signals and the like. The processor 114 carries out a variety of functions as is known in the art, as dictated by software and/or firmware instructions.
[0316] The computational device 113 may also include one or more data storage devices, such as a hard disk drive, flash memory drive, CD-ROM drive and/or other hardware capable of reading and/or storing information.
[0317] Software may be stored in the read-only memory of the computational device 113 and/or distributed on a removable memory device or other form of media capable of portably storing information. These storage media may be inserted into, and read by, devices such as the CD-ROM drive, the disk drive, USB drive etc.
[0318] The processor 114 may be coupled to a display, which may be any type of known display or presentation screen, such as LCD displays, LED displays, plasma display, cathode ray tubes (CRT), etc. A user input interface may be provided, including one or more user interface mechanisms such as a mouse, keyboard, microphone, touch pad, touch screen, voice-recognition system, etc.
[0319] The processor 114 may be coupled to other computing devices, such as the landline and/or wireless terminals via a network.
[0320] The computational device 113 may be part of a larger network configuration as in a global area network (GAN) such as the Internet, which allows ultimate connection to the various landline and/or mobile client devices.
[0321] For example, the processor 114 of the computational device 113 may communicate with one or more remote processors.
[0322] The computational device 113 can be for example a personal computer (PC), a tablet or even a sufficiently performing smartphone.
[0323] In case of a PC, preferably the computer runs with a Windows operating system, more preferably with Windows 10Pro or upper versions, and the hard disk drive is preferably of at least 256 GB.
[0324] With reference to
[0325] The method 500 is preferably carried out by the system 100 of
[0326] As already stated, the method 500 of the invention allows to easily acquire photographic images of a subject H having a plurality 110 of markers attached to the skin thereof at predefined anatomical landmarks of an anatomical structure of interest, and thereby allows to easily determine a “pattern of points” of such anatomical structure, based on which parameters of interest may be extracted.
[0327] In
[0328] In block 502, an uncalibrated image depicting a calibration reference is acquired. The uncalibrated image depicts in particular the support 116 carrying on its upper surface 118 the calibration reference CRef, constituted by the tags 120a-120d (shown in
[0329] The uncalibrated image is captured (block 501) by the camera 112 positioned with respect to the support 116 in a similar image capturing configuration as that of
[0330] Based on the uncalibrated image acquired in block 502, in block 504 an image calibration operation is carried out, through which an absolute calibrated reference system (x, y, z) is defined.
[0331] In the calibration operation of block 504, axes x, y are first defined based on the aligned tags 120a-120d (see in particular
[0332] Moreover, a [pixel] to [cm] conversion factor is determined based on the known resolution of the camera 112, and on the known distance d.sub.1, d.sub.2 between the tags 120a-120b; 120a-120d positioned on the support 116.
[0333] In step 506, a first calibrated image of a plurality 110 of markers 40a-40d applied over the skin of the subject H at the relevant body landmarks of the spine S is acquired.
[0334] The first calibrated image is captured (block 503) by the camera 112 positioned with respect to the subject H as in the image capturing configuration illustrated in
[0335] As stated above, the first calibrated image depicts a marker spatial arrangement in the first plane (x, z) substantially parallel to (or coinciding with) the median plane t.sub.1 of the support 116.
[0336] In order to obtain a good three-dimensional reconstruction of the spine, from which parameters of interest may be reliably estimated, each of the markers 40a-40d should be attached at the relevant body landmark precisely enough so that the contact point of the marker—substantially corresponding to the center of the contact surface 46 of the marker—matches as closely as possible the body landmark on which the marker is to be applied.
[0337] To this end, positioning of the markers 40a-40d is preferably carried out by qualified staff, or at least by specifically trained staff.
[0338] In block 508, a second calibrated image of the plurality 110 of markers applied over the skin of the subject H at the relevant body landmarks of the spine S is acquired.
[0339] The second calibrated image is captured (block 505) by the camera 112 in the image capturing configuration illustrated in
[0340] As stated above, the second calibrated image depicts a marker spatial arrangement in the second plane (y, z) substantially parallel to median plane t.sub.2 and perpendicular to median planet t.sub.1.
[0341] In a preferred embodiment, to pass from the image capturing configuration of
[0342] In this way, the camera 112 remains advantageously stationary and no adjustments nor resettings are necessary.
[0343] A person skilled in the art will understand that instead of rotating the subject H, it is equally possible to move the camera 112 with respect thereto.
[0344] In subsequent blocks 510 and 512, first and second initial positions of contact points of each marker 40a-40d with the respective body landmark 50a-50d of the subject H are determined respectively in the first and second calibrated images, acquired in blocks 506, 508 and each captured in a respective one of the perpendicular first and second planes (x, z; y, z).
[0345] After execution of blocks 510, 512, two sets of two-dimensional coordinates (X.sub.i-1, Z.sub.i-1) and (Y.sub.i-2, Z.sub.i-2) in the absolute coordinate reference system, related to the first and second initial positions of contact points of each marker, are obtained.
[0346] Coordinates (X.sub.i-1, Z.sub.i-1) and (Y.sub.i-2, Z.sub.i-2), in which index “i” is an integer spanning between 1 and the total number of markers (4 in the case illustrated in
[0347] The detailed operations carried out in blocks 510 and 512 are outlined hereinafter in connection with
[0348] In block 514, positions (X.sub.i, Y.sub.i, Z.sub.i) of contact points of each marker 40a-40d, expressed by three-dimensional coordinates in the absolute calibrated reference system (x, y, z) and also referred to as 3D coordinates, in contrast with the first and second 2D coordinates, are determined by combining, for each marker, the related first and second initial positions (X.sub.i-1, Z.sub.i-1) and (Y.sub.i-2, Z.sub.i-2) of contact points, according to the following formulas (1) to (3).
X.sub.i=X.sub.i-1 (1)
Y.sub.i=Y.sub.i-2 (2)
Z.sub.i=(Z.sub.i-1+Z.sub.i-2)/2 (3)
[0349] In detail, the 3D coordinate Z.sub.i is determined by averaging the common first and second 2D coordinates Z.sub.i-1 and Z.sub.i-2, whereas the other two 3D coordinates X.sub.i and Y.sub.i are set as the corresponding 2D coordinates X.sub.i-1 and Y.sub.i-2 respectively determined from the first and second calibrated images.
[0350] In subsequent block 516, the positions (X.sub.i, Y.sub.i, Z.sub.i) of contact points of all markers 40a-40d are fitted with a three-dimensional curve, which constitutes a three-dimensional pattern of the spine S of the subject H.
[0351] The fitting operation of block 516 is carried out by means of a suitable interpolation algorithm such as polynomial interpolation or spline interpolation. Preferably, cubic spline interpolations is used.
[0352] As known, given a set of n+1 different data points or nodes falling within a main numerical interval and dividing the main numerical interval in n sub-intervals, the cubic spline s(x) is the function defined piecewise by n third-order polynomials, each defined in each of the n sub-intervals.
[0353] The second derivative of each polynomial is set to zero at the endpoints of each of the n sub-intervals, to ensure a smooth curvature of the spline s(x) at the “junction” between each couple of polynomials.
[0354] In the case analyzed, the positions (X.sub.i, Y.sub.i, Z.sub.i) of contact points of the four markers 40a-40d are set as nodes, the first and the last position along the spine representing the ends of the main numerical interval.
[0355] Further data points may be added, within the main interval, so as to define narrower sub-intervals and thereby obtain a smoother curve.
[0356] By way of example, the spline s(x) may be defined by 999 polynomials defined in 999 corresponding sub-intervals delimited by 1000 points. Further data points besides the four positions of the markers may be found by setting up a matrix system.
[0357] The fitting operation of block 516 yields a three-dimensional curve that represents a pattern of the spine S of subject H, that will be referred to hereinafter as reconstructed curve or RC.
[0358] In subsequent block 518, parameters of interest are estimated from the reconstructed curve determined in block 516.
[0359] A parameter of particular interest that can be extracted from the reconstructed curve of the spine is the Cobb angle γ, as schematically illustrated in
[0360] As known in the art, the Cobb angle is defined as the greatest angle at a certain region of the vertebral column, conventionally measured in X-ray images of the spine in the coronal plane, measured from the superior endplate of a predetermine superior vertebra to the inferior endplate of a predetermined inferior vertebra. The Cobb angle allows to estimate the entity of bending disorders of the spine (e.g. scoliosis), both due to postural issues and to traumatic events.
[0361] The reconstructed curve RC in
[0362] First, to determine the Cobb angle of the reconstructed curve RC, changes of concavity along the reconstructed curve are preferably determined by finding the zero-derivative points (e.g. P1 and P2 in
f″=d.sup.2f/dx.sup.2=0 (4)
[0363] Thereafter, the equations of straight lines r.sub.1 and r.sub.2 orthogonal to the reconstructed curve RC at the zero-derivative points P1 and P2 are determined according to the following formulas (5) and (6) (in which the y-intercept is neglected for simplicity):
r.sub.1=m.sub.1x (5)
r.sub.2=m.sub.2x (6)
[0364] with m.sub.1=tan(ω), m.sub.2=tan(ϕ)
[0365] The values of angles ω and ϕ are determined based on the first derivative of the zero-derivative points P1 and P2.
[0366] Cobb angle γ is thus determined according to the following formula (7):
γ=arctan|(m.sub.1−m.sub.2)/(1+m.sub.1m.sub.2)| (7)
[0367] According to the knowledge in the art, scoliosis is generally associated to a Cobb angle of more than 10°.
[0368] Going back to
[0369] In particular, the reconstructed curve RC may be displayed as it is (in the form of a simple three-dimensional “line”) to appreciate a pattern of the spine S, or in more advanced applications it might be used to create a proper three-dimensional model of the spine S, including 3D-modeled bone structures.
[0370] Such three-dimensional model may be built, based on standard three-dimensional templates of the human spine (and possibly of neighbouring bone structures, e.g. the pelvis), by setting the reciprocal spatial arrangement of the various bones according to the pattern defined by the reconstructed curve RC.
[0371] The three-dimensional model can be built according to any suitable 3D modeling algorithm.
[0372] Examples of commercial 3D-modeling software that are nowadays available and could suitably be used to build a similar model of the spine are, i.e., GameStudio, Autodesk Maya, Autodesk Mudbox, Houdini, Cinema 4D, Modo, Autodesk 3Ds Max, ZBrush, Rhinoceros, Lightwave 3D, 3DCoat, Blender, Daz Studio Hexagon, Fusion 360, Houdini Apprentice, Wings3D, Rocket 3F, Sculptris.
[0373] Finally, in step 522 data related to the reconstructed curve RC, as well as the parameters extracted in block 518, are preferably stored in the memory of the computational device 113 and/or in a remote central database.
[0374] With reference to
[0375] The following steps are carried out for each i-th marker 40a-40d depicted in the first calibrated image, acquired in block 506.
[0376] Block 510 comprises a sub-block 524 in which, for each marker 40a-40d, one or more glyphs 20a-20f (see
[0377] The memory of the computational device 113 stores a database in which each glyph is associated with the face or group of faces on which it is applied/printed, along with information related to the geometry of the marker 40a-40d. For each marker 40a-40d in the first calibrated image, the recognition algorithm detects and recognizes one or more glyphs and associates the glyph to the face on which it is provided.
[0378] The predetermined glyph recognition algorithm can be implemented by any suitable commercial or non-commercial software for optical glyph recognition, preferably comprising the steps of: [0379] grayscaling the first calibrated image; [0380] detecting one or more glyphs by recognizing their quadrilateral dark shape emerging against the respective white faces of the marker on which they are applied; [0381] “binarizing” each detected glyph by dividing it into a grid of cells having same width and height (e.g. grid 30 of
[0383] Based on the number of glyphs recognized in the first calibrated image through the glyph recognition algorithm (block 524), in following blocks 526-532 the contact point of each of the markers 40a-40d is preferably determined through different geometrical calculations, according to the known geometry of the regular dodecahedral main body of the marker 40a-40d.
[0384] Relevant geometric features of the dodecahedral main body 12 of a marker 40 (see also
[0385] Based on the known length of the edge E of the regular dodecahedral main body 12 and on the known relevant angles thereof, the dimensions of the sides a, h.sub.1, b of the right-angled triangle 54 (cf. also
a=0.85×E (8)
h.sub.1=1.11×E (9)
b=1.40×E (10)
[0386] The same applies for analogous triangles construed on each face and on each edge of the dodecahedral main body 12 of the marker 40.
[0387] Still according to trigonometric calculations, the distance k between the centre of symmetry C.sub.g of the second lateral face 28 and the “lower vertex” V.sub.low, namely the intersection point between proximal extensions of oblique sides 25, 27 of a second lateral face 28, is determined according to formula (11):
k=2.23×E (11)
[0388] The same distance k is defined between the centre of symmetry C.sub.g of the first lateral face 26 and the “upper vertex” V.sub.up, namely the intersection point between distal extensions of oblique sides 29, 31 of a first lateral face 26.
[0389] These geometric features are exploited in the steps of the method 500 outlined below, for determining the contact point C of the marker 40a-40d with the respective body landmark.
[0390] Going back to
[0391] Preferably, the axes w.sub.1 and w.sub.2 are oriented parallel to sides of the glyph, whereas the axis w.sub.3 is oriented towards inside the marker.
[0392] In particular, for a first lateral glyph provided on a first lateral face 26 (
[0393] In block 528, the position of center of symmetry C.sub.m of the main body of each marker 110 is determined in the first calibrated image.
[0394] Block 528 preferably involves different operations based on the glyph(s) recognized, as detailed below.
[0395] If only the distal glyph provided on the distal face 24 is recognized, the center of symmetry C.sub.m of the main body of the marker is determined, in the first calibrated image, at coordinates (0, 0, 0) in the first relative reference system (w.sub.1, w.sub.2, w.sub.3), namely it is approximated to correspond to the centre of symmetry C.sub.g of the distal glyph on distal face 24 recognized.
[0396] If only one of the first or second lateral glyphs, provided either on a first lateral face 26 or on a second lateral face 28, is recognized in the first calibrated image, the center of symmetry C.sub.m of the main body of the marker is determined at coordinates (0, 0, h.sub.1) in the first relative reference system (w.sub.1, w.sub.2, w.sub.3).
[0397] If two glyphs are recognized in the first calibrated image, the center of symmetry C.sub.m of the main body of the marker is determined at the intersection point of the axes w.sub.3 of the first relative reference systems (w.sub.1, w.sub.2, w.sub.3) defined for each glyph.
[0398] If three glyphs are recognized in the first calibrated image, relative coordinates of the center of symmetry C.sub.m of the main body 12 of the marker are determined by averaging the coordinates of the intersection points between the axes w.sub.3 of the first relative reference systems (w.sub.1, w.sub.2, w.sub.3) of all the possible pairs of glyphs.
[0399] If four or more glyphs are recognized in the first calibrated image, relative coordinates of the center of symmetry C.sub.m of the main body 12 of the marker are determined by weight averaging the coordinates of the intersection points between the axes w.sub.3 of the first relative reference systems (w.sub.1, w.sub.2, w.sub.3) of all the possible pairs of glyphs.
[0400] In subsequent block 530, the position of the lower vertex V.sub.low of the marker, as described above with reference to
[0401] Block 530 involves different operations based on the glyph(s) recognized in the first calibrated image, as detailed below.
[0402] If only the distal glyph provided on the distal face 24 is recognized, the lower vertex V.sub.low of the main body of the marker is determined, in the first calibrated image, at coordinates (0, 0, 0) in the first relative reference system (w.sub.1, w.sub.2, w.sub.3), namely it is approximated to correspond to the centre of symmetry C.sub.g of the glyph.
[0403] If only one first lateral glyph provided on a first lateral face 26 is recognized in the first calibrated image, the upper vertex V.sub.up is determined at coordinates (0, k, 0) in the first relative reference system (w.sub.1, w.sub.2, w.sub.3). The lower vertex V.sub.low is then determined as the symmetrical point of said upper vertex V.sub.up with respect to the centre of symmetry C.sub.m of the main body of the marker.
[0404] If only one second lateral glyph provided on a second lateral face 28 is recognized in the first calibrated image, the lower vertex V.sub.low is determined at coordinates (0, k, 0) in the first relative reference system (w.sub.1, w.sub.2, w.sub.3).
[0405] If two first lateral glyphs provided on a first lateral face 26 are recognized in the first calibrated image, the upper vertex V.sub.up of the marker is determined at the intersection point of the axes w.sub.2 of the first relative reference systems (w.sub.1, w.sub.2, w.sub.3) defined for each of the two first lateral glyphs. The lower vertex V.sub.low is then determined as the symmetrical point of said upper vertex V.sub.up with respect to the centre of symmetry C.sub.m of the main body 12 of the marker.
[0406] If two second lateral glyphs provided on a second lateral face 28 are recognized in the first calibrated image, the lower vertex V.sub.low of the marker is determined at the intersection point of the axes w.sub.2 of the first relative reference systems (w.sub.1, w.sub.2, w.sub.3) defined for each of the two second lateral glyphs.
[0407] If the distal glyph, provided on the distal face 24, and one first lateral glyph, provided on a first lateral face 26, are recognized in the first calibrated image, the upper vertex V.sub.up is determined at the intersection point between axis w.sub.3 of the first relative reference system (w.sub.1, w.sub.2, w.sub.3) of the distal glyph and axis w.sub.2 of the first relative reference system (w.sub.1, w.sub.2, w.sub.3) of the first lateral glyph. The lower vertex V.sub.low is then determined as the symmetrical point of said upper vertex V.sub.up with respect to the centre of symmetry C.sub.m of the main body 12 of the marker.
[0408] If the distal glyph, provided on the distal face 24, and one second lateral glyph, provided on a second lateral face 28, are recognized in the first calibrated image, the lower vertex V.sub.low is determined at the intersection point between axis w.sub.3 of the distal glyph and axis w.sub.2 of the first relative reference system (w.sub.1, w.sub.2, w.sub.3) of the first lateral glyph.
[0409] All other cases are preferably treated as combinations of two or more of the cases above, and the “final” lower vertex V.sub.low is determined by weight averaging the position of V.sub.low obtained in each case above.
[0410] In the averaging or weight averaging operations described above, values above/below predefined thresholds and/or having a standard deviation higher than a predefined threshold can be discarded.
[0411] Lastly, in step 532, the first initial position of the contact point C (cf.
[0412] Once the contact point C is found in the first calibrated image, the corresponding first 2D coordinates (X.sub.i1, Z.sub.i1) in the absolute calibrated reference system (x, y, z) can be simply “read” in the calibrated image, since every position within the images captured from by the camera 112 and acquired by the processor 114 are mapped in the absolute calibrated reference system (x, y, z) as a result of the calibration operation carried out in block 504 described above.
[0413] Although block 510 and related sub-blocks have been described with reference to a marker 40, the same analysis applies, mutatis mutandis, to markers 10 illustrated in
[0414] Moreover, although the description above in connection with
[0415] In particular, the same description made above applies to block 512 by changing “first calibrated image” into “second calibrated image”, “first initial position” into “second initial position”, “first relative reference system” into “second relative reference system”, first 2D coordinates (X.sub.i1, Z.sub.i1) into second 2D coordinates (Y.sub.i1, Z.sub.i1), and so forth.
[0416] In addition, it should be underlined that although blocks 502-522 of the method 500 are illustrated and described in a sequence, the order of some blocks could be modified without departing from the scope of the invention.
[0417] For example, the calibration operation of block 504 could also be carried out after the uncalibrated image and the first and second calibrated images have all been acquired (blocks 502, 506 and 508), the order of acquisition of the three images being irrelevant.
[0418] Alternatively or in addition, each of blocks 510 and 512 of determining the first and second initial positions of the contact point of the markers could be carried out after the respective acquisition block 506, 508 of the first and second image.
[0419] Moreover, blocks 516-522, related to the operations of curve fitting, parameters estimation, display of the reconstructed curve and data storage, could be carried out in a different reciprocal order or even simultaneously.
[0420] Moreover, although the system 100 has been described as comprising a single computational device 113 comprising a processor 114, such system may comprise a second remote processor, part of a second remote computational device, located elsewhere.
[0421] In particular, in such embodiment (not illustrated) the processor 114 may be programmed to execute blocks 502-508 of method 500 of the invention and to further send the first and second calibrated images to the remote processor (not illustrated), which may be programmed to acquire the first and second calibrated images from the processor 114 and to further execute blocks 510-522 of the method 500.
[0422] The foregoing detailed description has set forth various embodiments of the devices, system and methods via the use of block diagrams, schematics, and examples. With particular reference to the method according to the invention, insofar as such block diagrams, schematics, and examples contain one or more functions and/or operations, it will be understood by those skilled in the art that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof.
[0423] However, those skilled in the art will recognize that the embodiments of the method disclosed herein, in whole or in part, can be implemented as one or more computer programs running on one or more processors similar to processor 114 described above (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more controllers (e.g., microcontrollers), as one or more programs running on one or more processors (e.g., microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and/or firmware would be well within the skill of one of ordinary skill in the art in light of this disclosure.
[0424] Those of skill in the art will recognize that many of the steps and algorithms set out herein may employ additional acts, may omit some acts, and/or may execute acts in a different order than specified.
[0425] In addition, those skilled in the art will appreciate that the mechanisms taught herein are capable of being distributed as a program product in a variety of forms, and that an illustrative embodiment applies equally regardless of the particular type of signal bearing media used to actually carry out the distribution. Examples of signal bearing media include, but are not limited to, the following: recordable type media such as hard disk drives, CD ROMs, digital tape, and computer memory.