Method and system for generating handwritten text with different degrees of maturity of the writer
10977422 · 2021-04-13
Assignee
Inventors
- Maria Cristina Carmona Duarte (Las Palmas de Gran Canaria, ES)
- Miguel Angel Ferrer Ballester (Las Palmas de Gran Canaria, ES)
Cpc classification
E04F11/1834
FIXED CONSTRUCTIONS
International classification
Abstract
The invention relates to a method and system for generating handwritten text with different degrees of maturity of the writer without needing to have the original text of the writer, thereby allowing the generation of text online (synthesizing temporal movement) and offline (generating the image of the handwriting), the simulation of real development in terms of the number of strokes and the velocity profile of the writer according to his or her progress, and the generation of handwriting of writers with possible neurological and motor disorders.
Claims
1. A method for generating valid synthetic handwritten text for synthetic writers with different degrees of maturity, wherein the different degrees of maturity are related to at least one of an age or learning progress of a particular synthetic writer, wherein points of a grid fix a trajectory of each character to be generated and the points of the grid are stored in a memory, the method comprising: (i) for each letter to be synthesized, selecting its n.sub.sl points making up its trajectory such that they allow performing paragraph (ii); (ii) selecting randomly a percentage E of initial points n.sub.sl that the trajectory of each letter has, wherein the percentage E is inversely proportional to the development or maturity of the writing, and the development or the maturity of the writing are related to either an age or learning progress of a writer; (iii) connecting each of the letters, forming the word by joining an end point of a preceding letter to an initial point of a next letter, making sure that both upper line and lower line of the grid are joined together; (iv) transforming the positions of the points for each trajectory into a temporal sequence of vectors, calculating an initial time instant corresponding to each vector depending on a preceding vector, wherein for every two consecutive points, a vector
2. The method according to claim 1, wherein step (ii) of the method further comprises: setting a rule on legibility of synthesized text; when the rule on legibility is not complied with, repeating the random selection of points again until the rule on legibility is complied with; wherein the rule on legibility of the text requires that at least one of the selected points coinciding with the upper line and at least one of the selected points coinciding with the lower line are maintained.
3. The method according to claim 1, characterized in that the predetermined equation of paragraph (vi) is determined by:
4. The method according to claim 1, characterized in that the predetermined equation of paragraph (vi) is determined by:
5. The method according to claim 1, wherein the predetermined sigmoid curve S of paragraph (iv) is approximated by (S(α,b,−c)), where the value of b is 0.06 and c is 65.
6. The method according to claim 1, wherein the initial time instant of each vector t.sub.0j is expressed as:
7. The method according to claim 1, wherein the method is performed by an online computer device, and the velocity profile is generated based upon either the age or the learning progress of the writer.
8. The method according to claim 1, further comprising: obtaining writing characteristics of a real person by means of adjusting parameters K.sub.μ, K.sub.t, K.sub.σ, K.sub.α and K.sub.D based on the real velocity profile; and obtaining the trajectory vectors from the real writing of the writer to be synthesized.
9. The method according to claim 1, characterized by defining the value of E from 100 to 20 to emulate child-to-adult development.
10. The method according to claim 1, wherein parameters ε.sub.D and ε.sub.t are related to learning or neurological health of an individual.
11. The method according to claim 1, further comprising: synthesizing patients with Parkinson's disease by means of increasing a time increment between signals |{right arrow over (ν)}.sub.j(t;t.sub.0j)| by means of increasing Kt in a manner proportional to the extent of the disease until the value thereof exceeds the width of said signal, wherein the vectors of the trajectory are designed in pairs, and, in each of the pairs, the direction of one vector offsets the direction of the preceding vector.
12. The method according to claim 1, further comprising: synthesizing patients with Alzheimer's disease by means of increasing the parameters related to the starting point stability of the signal representing each vector and an error in the trajectory (ε.sub.D, ε.sub.t).
13. The method according to claim 1, further comprising: synthesizing patients with amyotrophic lateral sclerosis by means of increasing the width of |{right arrow over (ν)}.sub.j(t;t.sub.0j)| by varying K.sub.σ∘K.sub.μ and reducing an amplitude (K.sub.D) thereof.
14. A non-transitory medium with instructions configured to be run by at least one processor, which instructions, when run by an electronic device, cause said electronic device to carry out the method for generating valid synthetic handwritten text for synthetic writers with different degrees of maturity, wherein the different degrees of maturity are related with at least one of an age or learning progress of a particular writer, the non-transitory medium comprising: instructions for storing points of a grid in a memory, wherein the stored points fix a trajectory of each character to be generated; instructions, for each letter to be synthesized, for selecting n.sub.sl points making up the trajectory of the respective letter; instructions for selecting randomly a percentage E of initial points n.sub.sl that the trajectory of each letter has, wherein the percentage E is inversely proportional to the development or maturity of the writing and the development or the maturity of the writing are related to either an age or learning progress of a writer, instructions for connecting each of the letters; instructions for forming the word by joining an end point of a preceding letter to an initial point of a next letter, making sure that both upper line and lower line of the grid are joined together; instructions for transforming the positions of the points for each trajectory into a temporal sequence of vectors; instructions for calculating an initial time instant corresponding to each vector depending on a preceding vector, wherein for every two consecutive points, a vector
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The drawings are presented so as to illustrate but in no way limit the invention. In the drawings:
(2)
(3)
(4)
(5)
(6)
DETAILED DESCRIPTION OF THE INVENTION
(7) Although the invention is described in terms of a specific preferred embodiment, it will be obvious for those skilled in the art that various modifications, rearrangements, and replacements can be made. The scope of the invention is defined by the claims attached hereto.
(8) The present invention allows providing a solution to the technical problem of synthesizing text similar to real text without needing to have an earlier original text of the writer, simulating real development in terms of the number of strokes and the velocity profile of the writer according to his or her progress and providing the possibility of varying parameters which may be related to neurological or motor parameters of human learning in a simple manner.
Detailed Description of a Preferred Embodiment of the Invention
(9) The present invention can be implemented in hardware, software, or a combination of both.
(10)
(11) Unlike in [Ferrer, M. A.; Diaz-Cabrera, M.; Morales, A., “Static Signature Synthesis: A Neuromotor Inspired Approach for Biometrics,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PP, no. 99, pp. 1, 1 (2015)], in the proposed method, it is considered that every two consecutive points of the grid is considered a vector
(12)
(
(13)
and the current vector
(14)
is also calculated, being defined as:
(15)
(16) With these previous calculations, it is considered that for every vector ({right arrow over (Q.sub.1Q.sub.j+1)}), parameters D.sub.j,t.sub.0j,μ.sub.j, and σ.sub.j of a signal are adjusted, said signal modelling the velocity profile and recommended to be a lognormal (or another which, by adjusting the parameters thereof, provides the same adjustment):
(17)
where μ and σ are the location and scale parameters, respectively, j indicates the number of the corresponding vector, and t.sub.0j is the initial instant where the start of the lognormal is located. The equations and premises that these parameters must have so that handwritten text that looks real and includes the requirements for writer development to be similar to real development can be synthesized, are described below.
(18) In this method, the value σ.sub.j which defines the width of each lognormal, is defined for each similar lognormal and determined by the equation:
σ.sub.j=K.sub.σ (3)
where K.sub.σ is a constant defined for each writer and is different for different writers. To generate development, the parameter K.sub.σ approximates 0.01 so as to simulate children's writing and it gradually varies to a value of 0.05 for adults in order to obtain results like the ones in
(19) The initial period of each lognormal t.sub.0j is expressed as:
(20)
where n.sub.sl is the number of points forming the trajectory of the letter, K.sub.t is a constant which can be set for each writer for setting the small scales of the letters and the mean velocity, K.sub.α is the maximum time of delay due to the angle, N(0,ε.sub.t) is a random variable which emulates dispersion and is inversely proportional to the writer's skill, and finally the delay introduced due to the angle between the vectors (α.sub.j) is determined by a sigmoid curve (S(α,b,−c)), where the value of b approximates 0.06 and c approximates 65 for the initial configuration, where said values can be increased or decreased up to 10% to better adjust the writer to be synthesized, and where:
(21)
(22) K.sub.t=0.04, K.sub.α=0.1, and ε.sub.t=+/−0.02 is set as the starting adjustment. These values can be adjusted to approximate the real parameters of a writer as explained for the angle. In this case, parameter K.sub.α would be adjusted with a greater time increment (Δt) (drawing on the bottom in
(23) Amplitude D.sub.j of the lognormal j is defined proportional to the distance between the points of the trajectory:
D.sub.j=K.sub.D(d.sub.act,j+N(0,ε.sub.D))/d.sub.refj=0,1,2, . . . ,n.sub.sl (6)
where d.sub.act,j is the distance between the points of the trajectory, Q.sub.j to Q.sub.j+1, d.sub.ref is the minimum distance between the points of the grid, K.sub.D is a constant dependent on each individual and can also be used for massive scaling of the letters in the cases in which K.sub.t is not sufficient. N(0,ε.sub.D) is a random value for each lognormal j. K.sub.D=d.sub.ref,j, and ε.sub.D=0.3 is set as the initial adjustment. These values can be adjusted by means of an approximation to adjust them to the real parameters of a writer, when this is the purpose thereof.
(24) The variable μ.sub.j is equal to a constant K.sub.μ, a constant which depends on the motor characteristics of the subject and is therefore different for each individual. In the experiment performed for obtaining
(25) Once all the parameters for each component of the trajectory {right arrow over (ν)}.sub.j(t;t.sub.oj) have been calculated, it is obtained by substituting parameters D.sub.j,t.sub.0j,μ.sub.j, and σ.sub.j in Equation 2. Finally, the final velocity vector is obtained by means of adding up each of the {right arrow over (ν)}.sub.j(t;t.sub.oj) based on [M. Djioua, R. Plamondon, A new algorithm and system for the characterization of handwriting strokes with delta-lognormal parameters, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31 (2009) 2060-2072]:
{right arrow over (ν.sub.n)}(t)=Σ.sub.j=1.sup.n.sup.
(26) Components x and y of vector {right arrow over (ν)}.sub.j(t) are adjusted by calculating the angle of each trajectory Ø.sub.j,
(27)
followed by substituting it into the equations to calculate ν.sub.x(t;t.sub.oj) and ν.sub.y(t;t.sub.oj) from {right arrow over (ν)}.sub.j(t;t.sub.oj) by means of (Equation 9) and (Equation 10).
(28)
(29) Finally, the Cartesian trajectory used for drawing the letter is obtained:
x(t)=∫ν.sub.x(t)dt (12)
y(t)=∫ν.sub.y(t)dt (13)
(30) To synthesize problems relating to the learning or neurological health of the individual, parameters ε.sub.D and ε.sub.t are modified, increasing their value the farther away the ideal individual to be synthesized is from the normal value.
(31) To synthesize patients with Parkinson's disease, the time increment between signals representing each vector (K.sub.t) is increased. In this manner, the more the disease affects the patient, the less the signals modeling each of the vectors will overlap one another. The vectors of the trajectory are designed in pairs, and in each of the pairs the direction of one offsets the direction of the preceding one (simulates agonist and antagonist muscle movement).
(32) To synthesize patients with Alzheimer's disease, the value of the parameters related to the starting point stability of the signal representing each vector and the error in the trajectory (ε.sub.D,ε.sub.t) is increased. The higher the value of these parameters, the greater the degree of degeneration will be due to the mentioned disease in adults.
(33) To synthesize patients with amyotrophic lateral sclerosis, the width of the signal representing each vector of the trajectory is increased by varying K.sub.σ∘K.sub.μ and reducing amplitude (K.sub.D). The greater the width and the smaller the amplitude, the greater the degree of degeneration will be due to the disease in adults.
(34) Alternative Method
(35) Instead of using Equation 1, a sinusoid can be used, where:
(36)
(37) The method would be the same as in the preceding method with the exception that in this case the value of σ.sub.j (Equation 3) would be multiplied by K.sub.Ds as follows:
σ.sub.j=K.sub.Ds(0.01+K.sub.σ) (14)
(38) Like K.sub.s, the value of K.sub.Ds depends on the writer. To simulate a real writer, the value is adjusted by comparing the synthesized value with the rest of the parameters and K.sub.D equal to 1 with the velocity signal obtained from a real sample. The real sample will be obtained from an image captured by means of a digitizing tablet on which the real writer will draw a line of 1 cm.
(39) Method for Temporal Development
(40) This section provides a detailed description of block (103) that is shown in detail in
(41) This method is based on the premise that, as learning progresses, the number of points used in the writing is smaller, and therefore the length d.sub.act,j is greater. To that end, a variable E defining the selected percentage of points for each individual is defined. Therefore, the number of outputs the system will have will be determined by L:
(42)
(43) This method is described in detail in
(44) Once the points of the grid have been calculated, the parameters are calculated with the preceding method.
EXAMPLES
(45) The invention will be illustrated below by means of tests performed by the inventors, which clearly show the efficacy of the invention.
(46) These specific examples that are provided serve to illustrate the nature of the present invention and are included only for illustrative purposes, so they must not be interpreted as limitations of the invention herein claimed. Therefore, the examples described below illustrate the invention without limiting the field of application thereof.
Example 1. Text Synthesized Automatically Offline
(47)
Example 1. Text Synthesized Automatically Online
(48)
(49) To obtain real samples, 20 children (from 5 to 10 years of age) from one and the same school were asked to write the letter ‘a’ on a Wacom digitizing tablet. The digitizing tablet is a WACOM Intuos 3 tablet with an Intuos 3 Grip Pen, a sampling frequency of 200 Hz, a resolution of 2540 dpi, and a work surface of 304.8 mm×228.6 mm. The same process was carried out with 10 adults over 18 years of age.
(50) The left column of
(51) The right column of