PRODUCT FOR GENERATING A THREE-DIMENSIONAL SHAPE AND ITS USE IN THE FABRICATION OF CUSTOM ORTHOSIS

Abstract

The invention provides an apparatus for use in determining the three-dimensional shape of an object, which apparatus comprises: an article, comprising a stretchable material, and a plurality of strain sensors positioned in contact with the material such that stretching of the material is detectable by the sensors; a computing device operatively coupled to the article and configured to receive output data from the sensors and to process the data to determine the three dimensional shape of the object. The invention also provides a method for providing a customized orthotic product using the apparatus.

Claims

1. An apparatus for use in determining the three-dimensional shape of an object, which apparatus comprises: an article, comprising a stretchable material, and a plurality of strain sensors positioned in contact with the material such that stretching of the material is detectable by the sensors; a computing device operatively coupled to the article and configured to receive output data from the sensors and to process the data to determine the three-dimensional shape of the object.

2. The apparatus according to claim 1, wherein the sensors are integrated or embedded within the material and/or positioned on the material surface.

3. The apparatus according to claim 1, wherein the sensors are provided in channels or the like formed in the material.

4. The apparatus according to claim 1, wherein the sensors are laminated on an elastomeric substrate, optionally wherein the substrate is in the form of a strip.

5. The apparatus according to claim 1, wherein the material is a textile and the sensors are positioned on, or integrated or embedded within, at least one yarn or fibre of the textile.

6. The apparatus according to claim 5, wherein the article is knitted or woven.

7. The apparatus according to claim 1, wherein the sensors (a) are positioned less than about 5 mm apart, such as about 4 mm, about 3 mm, about 2 mm or about 1 mm apart; and/or (b) have a height or thickness of less than about 0.02 mm, less than about 0.015 mm or less than about 0.01 mm, such as 9 μm, 8 μm, 7 μm, 6 μm or 5 μm; and/or (c) have a width of less than about 1.5 mm, less than about 1.0 mm, less than about 0.85 mm, less than about 0.80 mm, or less than about 0.75 mm, such as 0.6 mm, 0.5 mm, 0.4 mm, 0.3 mm, 0.2 mm or 0.1 mm.

8. The apparatus according to claim 5, wherein an individual yarn including sensors is encapsulated within a flexible cover, optionally a moisture or water-resistant flexible cover, optionally wherein the diameter of the individual yarn with cover is about 2.0 mm or less, preferably about 0.7 mm to about 1.0 mm, such as about 0.75 mm to about 0.95 mm or about 0.8 mm to about 0.9 mm.

9. The apparatus according to claim 1, wherein the sensors are coated to provide a washable article.

10. The apparatus according to claim 1, wherein the article is in the form of a sleeve adapted to snugly fit over a part of the object, for example a body part, such as a limb of a patient.

11. The apparatus according to claim 1, further comprising one or more sensors configured to detect at least one of pressure, compression, force, temperature, volume, blood oxygenation, pH, chemicals, skin surface moisture, flexion and rotation.

12. The computer-implemented method of generating a three-dimensional shape of an object using the apparatus according to claim 1, which method comprises the steps of: placing the article onto the object so that the material is stretched, at least in part; reading output from the sensors, and determining the three-dimensional shape of the object based on the output from the sensors; optionally further comprising fabricating the object in a three-dimensional form.

13. The method for providing a customized orthotic product, comprising: providing an apparatus according to claim 1; placing the article onto the object (body part) so that the material is stretched, at least in part; reading output from the sensors, and determining the three-dimensional shape of the body part based on the output from the sensors; using the determined three-dimensional shape of the body part to design a customized orthotic product.

14. The method according to claim 12, further comprising displaying the three-dimensional shape of the body part on a display.

15. The method according to claim 13, wherein the three-dimensional image of the shape of the body part is used to design a customized orthotic product by (1) fabricating the body part in three-dimensional form and designing the orthotic product using the fabricated body part, or (2) designing the orthotic product using the three-dimensional image of the body part, and fabricating the orthotic product in three-dimensional form from the designed image of the orthotic product.

16. The method according to claim 13, further comprising reading output from sensors at one or more target regions of the body part where force or deformation pressure is applied to hold, manipulate and/or correct the body part; optionally wherein the output is from sensors configured to detect at least one of pressure/compression, rotation, flexion and temperature; and using the output to design the orthotic product.

17. The method according to claim 16, wherein the force, deformation pressure, rotation and/or flexion to the target region(s) is applied by a practitioner's hands.

18. The method according to claim 12, wherein the three-dimensional shape of the body part is determined from sensor data using an algorithm/model that links inferred shape geometry to expected sensor reading; a loss function (L), measuring the degree of agreement between the sensor readings and the readings that would be expected based on the geometry of the inferred shape; and an optimisation function, iteratively modifying the inferred shape in order to minimise the value of the loss function.

19. The method according to claim 13, wherein the three-dimensional shape of the body part is determined from sensor data using an algorithm/model that links inferred shape geometry to expected sensor reading; a loss function (L), measuring the degree of agreement between the sensor readings and the readings that would be expected based on the geometry of the inferred shape; and an optimisation function, iteratively modifying the inferred shape in order to minimise the value of the loss function.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0046] FIG. 1 shows a sensing textile according to one embodiment of the invention and an enlarged view of a single yarn with sensors;

[0047] FIG. 2 shows illustrations of the approach (as described below) applied to a 3D cone where (a) shows the result of physical simulation (target) and the reconstructed algorithm (inferred), (b) shows a sensing textile according to one embodiment of the invention as applied to a 3D printed version of that same cone, and (c) shows the result of the reconstructed algorithm applied on sensor measurements of (b);

[0048] FIGS. 3 (a) and (b) show an example of a resistance temperature detector strip for use in one embodiment of the invention, and (c) shows the strip embedded within a textile yarn; and

[0049] FIG. 4 is a system architecture diagram showing examples of software and hardware that may be used in the apparatus and methods according to the invention.

DETAILED DESCRIPTION

[0050] The invention uses known techniques for fabricating personal three-dimensional printed orthotic products. The first step is to capture the anatomical geometry of the patient. In order to capture the personal three-dimensional geometry, an article according to the invention is placed onto or over the relevant body part in a stretched state. The second step is personal splint design; the physical structure is determined and modeled based on the three-dimensional geometry of the body part in combination with the additional information on target regions for a CAD designer to follow as instructions. The third step is three-dimensional printing of the personal orthotic. Any known three-dimensional printing technology can be used for the task.

[0051] With further reference to the geometric data gathering stage, the goal is to capture a patient's desired (relevant) anatomic region with the use of the article according to the invention placed in contact with the region. The relevant anatomic region of the patient is usually stabilized when the article is in position and data from the stabilization as well as the shape of the region is captured. A practitioner positions the patient's anatomy and provides pressure or force at target regions. A CAD designer follows these instructions during the CAD design stage. Alternatively, the CAD software can be modified to take the output from the sensors, for example, the pressure (compression), tension, force or bend sensors, as commands with determined anatomic locations and generate the design.

[0052] Commercially available CAD software, such as Autodesk Fusion 360, Rhinoceros 5, and Solid Works, can be used to import the patient's three-dimensional body surface geometry and to design a suitable body scaffold.

[0053] A preliminary stretchable sleeve with a grid of strain sensors was tested on cylindrical objects and the method and results are described below. The sleeve and the reconstruction algorithm produced a cone with a given radius to an accuracy of 0.44 mm.

[0054] Approach

[0055] The shape reconstruction approach of the invention is based on the integration of a dense mesh/grid of sensors capable of measuring localised textile deformation in the textile sleeve. In one implementation such mesh is a matrix structure. Other implementations may include a honeycomb arrangement of sensors, or other shapes suitable to measure the textile deformation. Sensors may measure elongation, bending, shear, or others. In one implementation, the sensors are strain sensors which measures the local elongation of the textile as the difference between its rest and stretched state. Similarly, with other modalities, the sensors measure the difference between the current deformation and the rest state.

[0056] Hereafter, “inferred shape” refers to the shape of the sleeve which is reconstructed by a method which processes the sensor readings. This inferred shape is represented in the method of the invention by nodes, each of which has a coordinate in 3D space, the ensemble of which defines the inferred shape in 3D.

[0057] The number and arrangement of nodes relate to the number and arrangement of sensors integrated in the sleeve. Depending on the sensor modality, sensors may be located in between nodes (i.e. on the edge between nodes), or at the location of the nodes. In one implementation, strain sensors measure localised textile elongation, and are placed in between nodes.

[0058] The shape reconstruction method used in the invention consists of three elements:

1) A model which takes the geometry of an inferred shape, defined as the ensemble of 3D nodes coordinates, and which allows to obtain the readings of sensors which would be expected from the geometry of the inferred shape, according to their modality and arrangement;
2) A loss function L, which measures the degree of agreement between all the sensor readings, and the readings that would be expected based on the geometry of the inferred shape and obtained following (1);
3) An optimisation function, which will iteratively modify the position of the nodes of the inferred shape, in order to minimise the value of the loss function.

[0059] (1) is implemented by physical modelling principles or using lookup tables. For example, the distance between nodes can be linked to the elongation of strain sensors located in between two nodes using physical modelling. For other sensor modalities, the appropriate corresponding modelling is used. For (2), a general form of a loss function L is given below, which measures the agreement between N sensor readings, with s.sub.inferred,i the expected reading of sensor i, and s.sub.target,i the actual reading of sensor i. A loss function value of 0 indicates a perfect match between the expected reading of the sensors and the actual reading.

[00001]$L=\underset{i=1}{\overset{N}{.Math.}}{\left(s_{\mathrm{inferred},i}-s_{\mathrm{target},i}\right)}^2$

[0060] (3) is an optimisation algorithm, which iteratively modifies the 3D coordinates of the nodes such that the function L is minimised. This optimisation algorithm can be a gradient descent algorithm, a population-based optimisation algorithm such as a genetic algorithm, or others.

[0061] In one implementation with strain sensors, the strain sensors are located on the edge between nodes. The length of the edges between nodes relates to the elongation measured by strain sensors. For illustration, FIG. 2(a) shows an elastic sleeve wrapping a cone, with the objective of reconstructing the shape of that cone based on the strain sensor readings. As the sensors are strain sensors, the model of (1) is a model which provides the expected reading of the strain sensor, based on the distance between nodes in the inferred shape (e.g. based on Hooke's law, or using some other approach such as lookup tables). This expected sensor reading is referred to as e.sub.inferred. The actual readings of the strain sensors are referred to as e.sub.target.

[0062] The loss function of (2) is defined as the sum of squared differences between the expected readings of the sensors, and the actual readings, for all the available sensors E:

[00002]$L=\underset{i=1}{\overset{E}{.Math.}}{\left(e_{\mathrm{inferred},i}-e_{\mathrm{target},i}\right)}^2$

[0063] The optimisation process of (3) starts from initial conditions of the inferred shape (i.e. initial shape), such as a cylinder with a diameter and length significantly larger than the expected target shape. A BFGS gradient descent algorithm (D. C. Liu et al., 1989, Mathematical Programming. 45, 503-528, Springer) was used to minimise the loss function L. The gradient descent computes a numerical approximation of the gradient by computing the partial derivative of the loss for a displacement of each of the nodes along each of X, Y and Z axes. Once the algorithm converges to an optima, the resulting mesh can be displayed and compared to the target. The inferred mesh after optimisation is indicated in FIG. 2(a) as is the mesh as a result of the physical simulation.

[0064] Sensing Textile

[0065] With reference to FIG. 1, the sensing textile (10) developed was formed of eight sensing rings (sr1, sr2, sr3, sr4, sr5, sr6, sr7, sr8), each one comprising eight strain sensors (12) positioned one after the other. For practical implementation purposes there are no sensing elements between adjacent rings.

[0066] The sensing rings (sr1-sr8) were formed by attaching commercially available conductive rubber cord stretch sensors (12) (length 20 mm, diameter 2 mm, Adafruit, USA) (henceforth referred to as rubber sensors) onto a textile sleeve (Rymora Calf Compression Sleeves, Rymora Sports, UK).

[0067] The rubber sensors (12) were connected to enameled copper wires (14) (diameter 0.19 mm) using a conductive epoxy (16) (CW2400, CircuitWorks, Chemtronics, Netherlands) at either end. Then, eight such sensors were attached onto an acrylic fettuccina yarn (18) (Yeoman Yarns, Leicester, UK) using sugru mouldable glue (Sugru Original formula, UK) to form the sensing rings (sr1-sr8).

[0068] The acrylic yarn was used as the carrier yarn and the copper wires were wrapped on it to prevent damage of the wires when the textile was stretched. The rubber sensors were positioned ≈5 mm apart from one another on the yarn.

[0069] The finalized structure had a rigid area (20) of 16.12±1.32 mm in between the active areas (21) of the rubber sensors (12). The next step was to attach these sensing rings onto the textile sleeve with a separation of 13.80±1.55 mm between each other. This was conducted in three steps, initially the rigid areas (20) of the sensing rings (sr1-sr8) were glued onto the surface of the textile sleeve using an adhesive (Evo-stick, UK). Then these rigid areas were sewn onto the sleeve and finally a layer of sugru was added on top of it to further secure the attachment. The textile sleeve had a slight conical topology, consequently, when the rings were attached onto the sleeve the distance between the active areas of the 1.sup.st and the 8.sup.th rubber sensor varied for each sensing ring. The 8.sup.th ring had a considerably smaller radius when compared to the rest. Therefore, the rigid area in between the 1.sup.st and 8.sup.th sensors was only 14.44 mm when compared to the rest (21.27±1.70 mm). The sensing textile (shown in FIG. 1) was connected to an interface hardware circuit comprising four 16 channel analogue multiplexers (CD74HC4067) connected to an Arduino Uno board. An algorithm was written in MATLAB™ to record the measurements from the sensors.

[0070] Calibration of the Smart Textile

[0071] The sensing sleeve (10) was calibrated using four three-dimensional printed cylinders. The cylinders had diameters ranging from 58 mm to 64 mm. For the calibration, the resistance of each of the rubber sensors (12) was measured when the sleeve was at rest. Then the sleeve (10) was put on a cylinder and the resistance measurements were obtained after 10 min. The change in resistance (ΔR/R0) was calculated using the resistance measurements, ΔR being the difference between the resistance at rest and when draped and R0 the resistance at rest. Thereafter the sensing sleeve (10) was taken off the cylinder and left 24 hours to recover. The sensors (12) were left to recover for this long since the preliminary response and recovery time experiments conducted had demonstrated that the rubber sensors took ≈3.1 h to completely recover once they were stretched to 50% strain and released. For these particular experiments, the rubber sensors were only stretched for <20% strain. The experiment was repeated up to three times for each of the cylinder diameters. The diameter of the cylinders along with the thickness of the sleeve were used to calculate the real circumference of the sensing rings. This was used to compute the elongation of each of the sensors. Thereafter, the polyfit function in MATLAB™ was utilised to create a first order polynomials using the resistance measurements and the elongation for each of the 64 sensors.

[0072] Application and Results

[0073] The sleeve draped two different three-dimensional printed cones (22). The elongation of the strain sensors was calculated utilising the resistance measurements capture from the strain sensors and the polynomial equations obtained. This elongation data was then added to the reconstruction algorithm and the cones were reconstructed as seen in FIG. 2(c). The actual radius of the cones in the locations where the sensing rings were positioned were measured using a vernier caliper for comparison purposes.

[0074] The real radius of the cones and the reconstructed radius from the sensing rings and the reconstruction algorithm are given in Table 1. The reconstruction algorithm approximates the shape of the cone, as displayed in FIG. 2(c). In cone 1, the average difference in between the radii computed by the reconstruction algorithm and the real measurements from the cone was 0.61±0.51 mm. This difference was mainly due to the measurements from the sensing rings 5 to 8. The rubber sensors in these rings did not elongate since the cone fitted loosely at the location of these rings. In the case of cone 2, the diameter of the cone was larger, therefore all the sensors were stretched ensuring a smaller average of difference of 0.44±0.22 mm.

TABLE-US-00001 TABLE 1 Real radii of the cones along with the radii computed by there construction algorithm. sr = sensing ring. sr 1 sr 2 sr 3 sr 4 sr 5 sr 6 sr 7 sr 8 (mm) (mm) (mm) (mm) (mm) (mm) (mm) (mm) Real radius 31.42 30.91 30.46 30.07 29.66 29.23 28.66 28.19 of cone 1 Reconstructed 30.85 31.01 30.38 30.04 30.13 30.35 29.93 29.49 radius of cone 1 Real radius 32.26 31.70 31.36 30.94 30.55 30.22 29.55 29.11 of cone 2 Reconstructed 31.38 31.25 30.77 30.42 30.30 30.36 30.06 29.35 radius of cone 2

EXAMPLE

[0075] Fabrication and Characterization of Strain, Pressure and Temperature Sensors

[0076] The first step is to fabricate strain, pressure and temperature sensors embedded within the fibres of a textile yarn. The sensors have length <5.00 mm, width <0.75 mm and a height <0.01 mm to be integrated within textile yarns. The strain and pressure sensors are initially fabricated using solution-based fabrication methods where stretchable substrates, such as latex, are infused with conductive networks of graphene/silver nanowires. Ideally, the fabricated strain sensors have a gauge factor >8. In the case of pressure sensors, a sandwiched structure is utilized to fabricate a capacitive sensor with a sensitivity preferably >0.05%/Pa. Their structure constitutes a latex layer positioned in the middle of two latex layers infused with a conductive network. For temperature sensors, standard microfabrication methods are used to fabricate Resistance Temperature Detectors (RTDs), where gold/chromium is patterned on a pre-stretched elastomer substrate. The RTDs preferably have a linear temperature sensitivity of >0.5Ω/° C. and a resistance >1 kΩ A commercial/production friendly way of fabricating a strain/pressure/temperature sensor is by patterning a conductive ink (graphene composite TPU ink) onto an elastomer substrate (preferably Ecoflex/Thermoplastic Polyurethane/latex). This technique will ensure the fabrication of sensor with a stretchability >400%. This method is implemented in the manufacturing of resistive strain sensors. For the pressure sensors, electrodes are printed on either side of a TPU/Ecoflex substrate to fabricate capacitive sensors. In the case of temperature sensors, RTDs are patterned by printing conductive electrodes onto the elastomer substrate. Several strain, pressure and temperature sensors are first patterned on an elastomer panel (35 mm by 15 mm). The panels are easy to fabricate and to handle. Thereafter the sensor panel is passivated using an epoxy such as SU-08.

[0077] Equipment: A class 1000 clean room is equipped with a thermal evaporator (BOC Edwards FL400), spin coater (Karl Suss RC 8) and UV mask aligner (Karl Suss) which is required for the fabrication of the temperature sensors. It also has a hot plate (IC20 chilling/heating dry bath Ecotherm) and stirrers needed for the synthesis of pressure and strain sensors. A high performance 3D printer (Optomec, Aerosol Jet HD) is used for the fabrication of the sensors. The printer is equipped with fine 20 μm print heads which assist in the fabrication of small sensor strips. A Stratasys Objet30 Prime 3D printer (PolyJet) has a precision of 36 μm.

[0078] Integration of Flexible Sensors into Textile Yarns and Textile Sleeve Fabrication

[0079] With reference to FIG. 3, the integration of these sensors within the textile yarn requires the elastomer panels to be cut into sensors strips (24) of length <5.00 mm and width <0.75 mm. A laser cutter is used to cut the panels onto sensor strips. The strips are wired onto the interface hardware. The sensors are connected using low resistance enameled copper wires to ensure that there will be no overlapping of signals between the wires. These wires are connected to the elastomer structure using a silver conductive epoxy. The sensors are positioned on a carrier textile yarn (25) afterwards and encapsulated with a flexible PDMS adhesive. The encapsulation ensures the washability and biocompatibility of the fabricated device. The carrier yarn and the adhesive layer provide additional mechanical strength to the strips. The enameled copper wires are wrapped around the carrier yarn to prevent damage due to mechanical strain generated during the usage of the stretchable sleeve. The final step is the covering of the carrier yarns containing the flexible sensor strips with textile fibres (FIG. 3(c)); commercially available yarn covering machines are used to achieve this. The textile cover (26) ensures that the sensors remained hidden and the feel of the textile is not altered. Suture braiders and knit braiding machines can be used to provide covers for flexible sensors. Once covered with textile filaments, the final yarn has a diameter ranging from 0.80 mm-0.90 mm. Preliminary research has demonstrated the possibility of this integration technique. Eight strain/temperature/pressure sensors are positioned <5.00 mm apart within one textile yarn. Eight yarns are positioned one after the other separated <5.00 mm. This integration technique ensures that standard textile manufacturing processes such knitting, weaving and embroidery can be used for the fabrication of the sleeve. The sleeve is fabricated on a flatbed knitting machine. Once the yarns and the sleeve are fabricated, they are characterized against commercially available sensors. The reliability and the effects of hysteresis on them is evaluated by conducting cyclic tests on a Zwick/Roell machine.

[0080] Equipment: Laser cutters (techsoft lasercam) are used for the precise cutting of polymers and fabrication of the sensor strips. A prototype textile yarn covering machine is used to provide a prototype cover for the initial samples. For a more uniform yarn cover, knit braiders (RIUS MC braiding machine) and a suture braider are used. The fabrication of the sleeve is done on the flat bed knitting machine (Silver Reed SK840). The testing and characterisation of the yarns and sleeve is conducted using multimeters, parameter analysers, signal generator, hot plates, oscilloscope, etc.

[0081] Design and Assembly of the Interface Hardware and Data Processing Software

[0082] With reference to FIG. 4, an interface hardware device (30) captures and conditions the sensor measurements. This device (30) transmits the data to a smartphone/personal computer (32) for further processing. The hardware comprises two CMOS multiplexers (32-channel) attached to a modified BlueSense device. Bluesense is a Bluetooth™ enabled microcomputer. It is battery powered and was chosen mainly due to its minute size (dimensions 30 mm by 30 mm). Nevertheless, it has limited analogue input channels, therefore the sensors are connected through a multiplexer.

[0083] Equipment: BlueSense microcomputer, a PCB milling machine (LPKF) and equipment electronics (soldering stations, reflow ovens, etc.) to fabricate conventional interface and multiplexing boards for BlueSense.

[0084] Design and Development of a Consumer Friendly Software System for Data Processing and Real Time Visualization.

[0085] A software developed for the acquisition and processing of data fetched from the textile body shape sensor devices, is able to perform an accurate real time 3D reconstruction. Compatibility and portability are mandatory requirements to support mobile and portable personal devices such as laptops, smartphones, or tablets with Android operative systems.

[0086] To satisfy high demanding time and space computational requirements, the software system architecture is a distributed Service Oriented Architecture (SOA), as shown in FIG. 4. As such, it includes a persistence layer for data storage (34) and a web service business logic component (36). This approach facilitates a scalable and modular approach to minimize the impact of future requirements or change requests. The textile body shape sensor is connected to the BlueSense microcomputer (30) to read and transmit the data. The BlueSense equipment establishes communication with a portable Android device (32). The latter have multiple responsibilities, among which are offering control and data presentation interface to the therapist through a graphical Human-Computer Interface (HCI). This interface exposes simple and advanced modes to facilitate users with small or no experience with computer aided design (CAD) software. In addition, the Android device delegates computing demanding and data persistence tasks to a Microsoft IIS web service (38) which runs the core business logic. Among the responsibilities of the server is the authentication and session management, as well as the processing of data models in available formats, such as STL, or XYZ, that will be compatible with commercial CAD software (e.g. Solidworks), through a SOA web service API. Additionally, the web server application is responsible for computing tasks including the reconstruction algorithms to overcome the computing limitations of portable devices. Such extensible architectural approach also enables the easy implementation of 3D format converters as internal or third-party plugins. To better support multiple Computer Aided Manufacturing (CAM) devices (40), such as Computer Numerical Control (CNC) machines, an adapter interface component (42) has been designed and linked to the core business logic (36).

[0087] It should be understood that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the present invention and without diminishing its attendant advantages. It is therefore intended that such changes and modifications are covered by the appended claims.