METHOD TO CONTROL THE TIMBRE OF A TARGET STRINGED INSTRUMENT IN REAL-TIME
20180190251 ยท 2018-07-05
Inventors
- Michele Benincaso (Johanneshov, SE)
- Agostino De Angelis (Lucera, IT)
- Carlo Fischione (Stockholm, SE)
- Luca Turchet (Copenhagen, DM)
- Stephano Zambon (Nacka, SE)
Cpc classification
G10H1/0058
PHYSICS
G10H2230/075
PHYSICS
G10H2210/265
PHYSICS
G10H3/186
PHYSICS
International classification
Abstract
A method and process to set the parameters of a Control Algorithm and to synthesize and generate a variety of sounds and vibration that are normally not available on a specific acoustic stringed instrument are disclosed. The technical problems of generating a different timbre, sound and vibration of a stringed acoustic instrument (Controlled Instrument) in real time are solved. The invention describes a process (Cloning Procedure) where specific set of parameters needed to imitate a target instrument are defined; The imitation of the Target Instrument's timbre on the Controlled Instrument is performed by the Control Algorithm which employs two digital Linear-Time-Invariant systems to drive the actuators based on by controlling the actuators with a couple of digital Linear-Time-Invariant systems that receive the vibration signal from the measurement apparatus. The process can be optionally deactivated, thus having the Controlled Instrument acting as a traditional acoustic instrument.
Claims
1) A method and a process to shape and control in real-time the acoustic response of a controlled instrument such to obtain a desired timbre and/or tone specified by a given target acoustic response, such method characterized by: a) a Controlled Instrument, preferably a stringed musical instrument capable of producing sound waves comprising a radiating body capable of vibration (101), a bridge (103), and strings (104). Such radiating body (101) divided into a principal part (201) and a secondary tonal chamber (202), such secondary tonal chamber being of smaller dimensions than the principal part (201) and being mechanically loosely coupled from the rest of the instrument; b) a mechanical frequency response H.sub.c (402) and an acoustic frequency response A (409) originated by the principal part (201) of such Controlled Instrument; c) a measurement apparatus s.sub.1 (203), placed in the principal part of the body of the Controlled Instrument (201) in a position close to the strings bridge (103), and such measurement apparatus (203) capable of reading the vibration of such Controlled instrument (201) and converting said vibration to an electronic signal; d) a tonal acoustic chamber (202), characterized by an acoustic frequency response A (406), which is mechanically independent and insulated from the rest of the body of the Controlled Instrument (201); e) number 2 force actuators (204) (205) or other comparable vibratory devices such as moving magnetic actuators or piezoelectric transducers capable of providing mechanical excitation coupled to said radiating body (201), the first force actuator a.sub.1 (204) placed in the principal part of the body (201) close to the measurement apparatus (203) and whose frequency response being characterized by the function H.sub.1 (403); the second force actuator a.sub.2 (205) placed in the secondary tone chamber (202) and having an acoustic frequency response A.sub.t (406); both actuators (204) and (205) being in communication with a controller (209) and configured to receive electrical signals and alter the vibration of said radiating bodies at the actuators locations (204) and (205); f) a controller (209) in communication with the measurement apparatus (203), such controller including a processor (207) to process the measured electrical signals (410) in accordance with a real-time control system (401) which produces output electrical signals according to the implementation of two Linear Time Invariant discrete-time systems K.sub.1 (404) and K.sub.2 (407); wherein such processor (207) includes at least one of the devices selected from the group consisting of: a microprocessor, a microcontroller, or an application specific integrated circuit; g) a mathematical model (401) of the controlled instrument, composed by the serial connection of the system described by the frequency response function H.sub.c (402) and the system defined by the parallel connection of A.sub.c (409) and the product of A.sub.t (406) and K.sub.2 (407) and having a feedback loop placed at the output of the measurement apparatus s.sub.1 (203) defined by the serial connection of the frequency response functions K.sub.1 (404) and H.sub.1 (403); h) an optimization procedure (601) which synthetically sets the pair of discrete time Linear Time Invariant (LTI) systems K.sub.1 (404) and K.sub.2 (407) according to a series of algebraic passages in such a way that the weighted squared error E (604) in the frequency domain between the desired response D (602) and the response of the controlled system G (401) is minimized; such algebraic passages specified in the following order: h.1. compute the parameters of the Linear Time Invariant system K.sub.2 (407) independently from K.sub.1 (404), as a result of the assumption of claim 1.c), in order to minimize the error E (604) when the contribute of the system K.sub.1 (404) in the feedback loop is assumed to be null using one of the known optimization techniques such as, and not limited to, the Least Square Method; h.2. subsequently compute the parameters of the Linear Time Invariant system K.sub.1 (404) starting from the resulting value of K.sub.2 (407) obtained from the algebraic passage of h.1, and including the contribution of the feedback loop, using one of the known optimization techniques such as, and not limited to, the Least Square Method;
2) A Cloning Procedure consisting of a method and a process as in claim 1, wherein the target acoustic response D (602) might be measured from a given acoustic stringed instrument or Target Instrument (501) by means of one of the known acoustic measurement techniques such as, and not limited to, the wire break method.
3) A method and a process as in claim 1, which can be activated or deactivated independently from the usage of the Controlled Instrument (i.e. the Controlled Instrument will play as a standard acoustic instrument when the process is deactivated and generate the desired timbre when the process is activated).
4) A method and a process as in claim 1, wherein the optimization procedure of point h) is performed by simultaneously optimizing the acoustic responses of all the strings of the Controlled Instrument according to a set of target acoustic responses, one for each string.
Description
3 DESCRIPTION OF THE DRAWINGS
[0008]
[0009]
[0010]
[0011]
[0012]
[0013]
4 DETAILED DESCRIPTION
[0014] The method and process of the present invention is described by referring to the figures, which are just an exemplification of the preferred execution and do not limit the invention to just such forms of representation.
[0015] The Target Instrument is any instrument composed of a radiating body (i.e., a soundboard (101) and/or a sound box (102)), a bridge (103), and strings (104) (see
[0016] The Controlled Instrument is composed of the following components (see
[0024] The parameters of the Controlled Instrument are calibrated using a calibration setup that consists of the following items (see
4.1 Mathematical Modelling of the Controlled and Target Instruments
[0030] In this section we characterise the mathematical modelling of the Controlled and Target Instruments, which is the essential preliminary step to specify the Control Algorithm in the next section.
[0031] Following a classic approach in control systems theory, the input/output relationships of the Controlled Instrument, and the Target Instrument, are detailed respectively in
[0032] The physical variables explicited on the signal branches are the following: [0033] U() (408) is a vector whose components are the force signals generated by the N.sub.s strings (409) of the instrument as the result of the excitation of the human player and acting on the bridge that couples the strings to the soundboard. [0034] V() (410) is the velocity of the soundboard in the proximity of the bridge measured by sensor s.sub.1 (203). [0035] Y.sub.c() (411) is the acoustic pressure generated by the principal part of the body instrument as measured by sensor s.sub.2 (301). [0036] Y.sub.t() (412) is the acoustic pressure generated by the secondary tone chamber as measured by sensor s.sub.2 (301). [0037] Y() (413) is the total pressure measured by sensor s.sub.2 (301) as the combined action of the principal body and the tone chamber.
[0038] The frequency response functions corresponding to mechanical and acoustical parts of the Controlled Instrument and the estimation procedures for their parameters are the following: [0039] H.sub.c() (402) is the mechanical impedance of the instrument body to the exciting action of the strings. Mathematically it is a row vector of dimension 1N.sub.s whose columns are the FRFs from the forces generated by each string and the velocity measured at sensor s.sub.1 (203). The FRFs of each column can be measured applying the wire break technique (Woodhouse 2004) to string having index n, 1nN.sub.s, while other strings are temporarily removed from the instrument. The recorded velocity signal measured at sensor s.sub.1 (203) is taken as the impulse response h.sub.c,n(t) and its Fourier Transform H.sub.c,n() is the n-th column of H.sub.c() (402).
[0040] H.sub.1() (403) models the combined action of the response of the actuator a.sub.1 (204) and the mechanical impedance of the instrument body excited in the position of the actuator. It can be measured by feeding the actuator a.sub.1 with a test signal such as a Logarithmic Sinusoidal Sweep (LSS) or a Maximum Length Sequence (MLS), recording the output signal measured with sensor s.sub.1 (203) and using standard system estimation techniques such as, and not limited to, Least Squares Optimization. [0041] A.sub.c() (405) models the acoustic radiation impedance of the main body of the instrument. It can be measured by feeding the actuator a.sub.1 (204) with a test signal (LSS or MLS) and recording the output recorded by pressure sensor s.sub.2 (301). [0042] A.sub.t() (406) is a lumped model of the response of the actuator a.sub.2 (205), the mechanical impedance and the acoustic radiation impedance of the separate tone chamber. It can be measured by feeding the actuator a.sub.1 (204) with a test signal (LSS or MLS) and recording the output recorded by pressure sensor s.sub.2 (301). [0043] K.sub.1() (404), K.sub.2() (407) are the FRFs of two separate computational blocks that represent the digital LTI systems implemented by the algorithms in the CPU1 (207).
[0044] With basic algebra manipulations, the overall matrix of FRFs G() (502) of the Controlled Instrument system can be derived as
The n-th component of G() (502), referred to as G.sub.n(), models the FRF from the n-th string to the pressure sensor s.sub.2 (301).
4.2 Control Algorithm
[0045] The Control Algorithm is based on a mathematical model of the Controlled Instrument we described in the previous section, which is the acoustic system under control. Such an algorithm is representable as a pair of discrete-time Linear Time Invariant (LTI) systems that are specified by K.sub.1() (404), K.sub.2() (407), respectively, and that are implemented by a microprocessor. The algorithm's behaviour consists of two main parts: (i) the creation of the resonances that are not present among those naturally producible by the Controlled Instrument (i.e., those resonances that could not be produced without using a Control Algorithm); (ii) the cancellation of the resonances that are present among those produced by the Controlled Instrument.
[0046] Since there are many ways to choose K.sub.1() (404), K.sub.2() (407), one may have several instances of the Control Algorithm. Each instance has its own implementation complexity. Finding a set of parameters for an instance of the Control Algorithm can be seen as an optimization problem, where the goal is to minimize in the frequency domain the weighted squared error E (604) between the Controlled Instrument's frequency response G() (401) and the desired frequency response D() (602). The weighting function W() 605 is arbitrarily chosen using e.g. a psycho-acoustic function that tries to give more importance to the frequency region that are most important for the human ear. In the following, the most general instance is specified (the Multi-Objective Control Algorithm), and then some special cases are provided.
4.3 Multi-Objective Control Algorithm
[0047] Let D() (602) be a desired FRF, namely a FRF that the Controlled Instrument has to exhibit so that it can produces the desired vibrations. The algorithm consists in specifying the control blocks K.sub.1() (404), K.sub.2() (407) so that the FRF of the Controlled Instrument G() (401), specified in Eq. (4.1), is as much close as possible to the desired FRF D() (602). Since the FRF is a matrix of complex numbers, we need to specify in which sense the two FRF are made equal. This is complicated by that we are dealing with complex numbers.
[0048] Let M.sub.D() be the matrix whose entries are the modules of the entries of the matrix D() (602), and let .sub.D() the matrix whose entries are given by the phases of the entries of the matrix D() (602). Analogously, let M.sub.G() be the matrix whose entries are the modules of the entries of the matrix G() (401), and let .sub.G(W) the matrix whose entries are given by the phases of the entries of the matrix G() (401). Moreover, let .Math. be any induced matrix norm, for example 1-norm, 2-norm, Frobenius-norm, oc-norm, max-norm, or min-norm, to mention some of the possibilities.
[0049] The most general way consists in choosing K.sub.1() (404), K.sub.2() (407) by a multi-objective optimisation so that the integral over the frequency domain of a weighted squared induced norm of the matrix difference among M.sub.D() and M.sub.G() is as small as possible, while the integral over the frequencies of a weighted squared induced norm of the matrix difference among .sub.D() and .sub.G() is as small as possible. Additionally, this optimisation has to ensure that the choice of K.sub.1() (404), K.sub.2() (407) give a FRF matrix G() must be BIBO (Bounded-Input, Bounded-Output) stable. Otherwise, the resulting system would present self-sustained oscillations due to errors in feedback control, which are usually referred to as Larsen effect by musicians. The constraint can be satisfied by exploiting well-known results in control systems theory concerning poles and zero placements of FRF. Formally, we have the following multi-objective optimisation problem:
In this optimisation problem, the weighing matrixes W.sub.M() and W.sub.() are chosen arbitrarily. For example, they can be a psychoacoustic weighting function that gives more importance they can give more importance to the frequencies important for the human hearing system, such as A-weighting, ITU-R 468 or similar functions. The decision variabler of the optimisation problem are K.sub.1() (404), K.sub.2() (407). In the problem, .sub.max 229 is the maximal frequency of interest for the application, usually close to the human hearing frequency limit (e.g. 20,000/2 rad/s). Note that in the optimisation problem we have two objectives: the simultaneous minimisation of the module and the simultaneous minimisation of the phases. The solution of the optimisation problem can be obtained by any solution method for multi-objective optimisation. This should not be a problem, since the solution can be achieved off-line. Nevertheless, in the following, we present some other approaches that are of reduced computational complexity.
[0050] A computationally more affordable way to solve optimisation problem (4.2) is via a scalarizarion procedure that leads to a Pareto optimisation as follows: First, we define a secularised cost function weighted by the Pareto coefficient 01:
p(K.sub.1(),K.sub.2())=.sub.0.sup..sup.
Then, the Control Algorithm parameters are give by the solution to the following optimisation problem:
min.sub.K.sub.
subject to BIBO stability of G().(4.6)
Note that in this optimisation problem, the choice of the coefficient is left to the implementer. For example, one could even choose =1 so to give no relevance to the phase minimisation. Alternatively, it can be done by constructing the standard Pareto trade-off curve and looking for the knee-point of the courve. Finally, observe that if optimisation problem (4.2) is convex in the decision variables K.sub.1() (404), K.sub.2() (407), then the optimal solution returned by (4.5) is identical to the one returned by (4.2). However, if problem (4.2) is non convex, then the solution of problem (4.5) is a feasible solution for problem (4.2) and in general is sub-optimal for problem (4.2).
4.4 Multi-Objective Sub-Optimal Control Algorithm
[0051] The methods to determine the values of K.sub.1() (404), K.sub.2() (407) can be computational demanding. Here, we describe a sub-optimal method that is of easier implementation. This method is sub-optimal compared to the more general method given by optimisation problems (4.2) and (4.5).
[0052] Once a desired FRF D() (602) is set, the Multi-Objective Sub-Optimal Control Algorithm consists in finding the control blocks K.sub.1() (404), K.sub.2() (407) that simultaneously minimise the squared error between the spectral magnitudes of the controlled and target FRFs, where the errors are defined component-wise:
E.sub.n=.sub.0.sup..sup.
where .sub.max is the maximal frequency of interest for the application, as for the previous problems, and W.sub.p() is a psychoacoustic weighting function that gives more importance to the frequencies relevant for the human hearing system, such as A-weighting, ITU-R 468 or similar functions. The optimization is subjected to the constraint that the resulting system defined by G() (401) must be BIBO (Bounded-Input, Bounded-Output) stable. The optimisation problem is
As for the previous section, the solution to this problem can via standard multi-objective optimisation methods. A computationally simple method of finding a feasible solution, which is optimal if the problem is convex, is by Pareto scalarization, where the solution to (4.8) is achieved by solving the following Pareto optimisation problem
Note that in this problem, the choice of the Pareto weighting coefficients .sub.i is left to the implementer. Alternatively, one can draw the usual Pareto trade-off curve and choose the .sub.i that give the knee-point.
4.5 Multi-Objective Heuristic Control Algorithm
[0053] Here we propose a simpler method to determine K.sub.1() (404), K.sub.2() (407). The method is heuristic in the sense that it is not analytically derived from the approach of the previous section, although it is inspired from it. Here, the determination of the values of K.sub.1() (404), K.sub.2() (407) consists of two steps. First, the parameters of the controller K.sub.2() (407) are estimated exploiting that the actuator a.sub.2 (205) is placed in an independent tone chamber. In this way it is possible to reproduce, in the Controlled Instrument, the acoustic resonances that are given by D() (602) but not in the Controlled Instrument itself when both the actuators are not active. In a formal way, the feed-forward FRFs of the Controlled Instrument when only actuator a.sub.2 (205) is active, is defined as:
{tilde over (G)}.sub.2,n()=[A.sub.t()K.sub.2()+A.sub.c()]H.sub.c,n().(4.14)
In the first step, an optimization problem is posed to find only the controller K.sub.2() (407) that minimize the weighted target error functions E.sub.t,n defined as
E.sub.t,n=.sub.0.sup..sup.
where the exponent .sub.t>1 controls the weight given to the frequency points where the spectral magnitude |D.sub.n()| is larger than |{tilde over (G)}.sub.2,n()|. In other words, the error E.sub.t,n 282 is subjected to an additional weight that gives more relevance to the frequency points where target response has resonances that are not present in the Controlled Instrument's response. At the same time, less effort is spent trying to suppress resonances that are present in the Controlled Instrument but not in the Target Instrument. Formally, the optimisation problem posed to determine K.sub.2() (407) is
As for the previous section, the solution to this problem can via standard multi-objective optimisation methods. A computationally simple method of finding a feasible solution, which is optimal if the problem is convex, is by Pareto scalarization, where the solution to (4.16) is achieved by solving the following Pareto optimisation problem
[0054] Let denote by K.sub.2*() the solution to this problem, namely K.sub.2*() is the transfer function of the calibrated controller that drives the actuator a.sub.2 (205) computed as the result of the first step of the algorithm.
[0055] If we now use K.sub.2*(), the overall FRFs of the Controlled Instrument depends now only on K.sub.1() (404), and results defined as
Then, we can determine K.sub.1() (404) by formulating an optimisation problem that minimizes the overall resulting error:
E.sub.r,n=.sub.0.sup..sup.
where 0<.sub.r<1 is the exponent that weights the effort towards the suppression of the unwanted resonances in the Controlled Instrument's response and, at the same time, simplify the task of designing a controller that maintains the constraint of BIBO stability. Formally, the optimisation problem posed to determine K.sub.1() (404) is
As for the optimisation problem of the first step, the solution to this problem can be done via standard multi-objective optimisation methods. A computationally simple method of finding a feasible solution, which is optimal if the problem is convex, is by Pareto scalarization, where the solution to (4.24) is achieved by solving the following Pareto optimisation problem
Let denote by K.sub.1*(a)) the solution to this problem, namely K.sub.1*(a)) is the transfer function of the calibrated controller that drives the actuator a.sub.1 (204) computed as the result of the second step of the algorithm.
4.6 Cloning Procedure
[0056] The Cloning Procedure provides the Control Algorithm with the parameters that regulate actuators of the Controlled Instrument when the Control Algorithm allows the reproduction of the vibrations of the Target Instrument. Otherwise, the Control Algorithm can have its parameters set so that the Controlled Instrument can reproduce any desired vibration. The Control Algorithm's behaviour for cloning a Target Instrument consists of two main parts: (i) the creation of the resonances that are not present among those naturally producible by the Controlled Instrument (i.e., those resonances that could not be produced without using a Control Algorithm), but that are present in the Target Instrument's timbre; (ii) the cancellation of the resonances that are present among those produced by the Controlled Instrument, but that are not present in the Target Instrument's timbre.
[0057] The objective of the Control Algorithm for cloning a Target Instrument is the imitation of the acoustic properties of a given target acoustic musical instrument by finding a proper set of parameters for the controllers K.sub.1() (404), K.sub.2() (407). A block diagram for the model of the Target Instrument is presented in
[0058] The Cloning Procedure is then finalised by imposing that D() (602)=G*() (502) and applying any Control Algorithm of the previous section.
4.7 References Cited
[0059] US2002/0005108A1 [0060] PCT/GB2000/000769 [0061] U.S. Pat. No. 6,320,113 B1 [0062] U.S. Pat. No. 8,389,835 B2 [0063] Woodhouse, Jim. Plucked guitar transients: Comparison of measurements and synthesis. Acta Acustica united with Acustica 90.5 (2004): 945-965.