Abstract
The invention concerns a digitizing ASIC (3) for an ultrasound scanning unit (2) of an ultrasound system (1), comprising: an array of analogue-to-digital converters (31) adapted to receive ultrasound signals acquired with an ultrasound transducer array and to convert the ultrasound signals into digitized ultrasound data; a memory module (32) operably coupled to the array of analogue-to-digital converters (31) and adapted to store the digitized ultrasound data; a transmitter operably coupled to the memory module (32) and adapted to transfer the digitized ultrasound data stored in the memory module (32) to a remote interface unit (6); and a controller (33) adapted to receive control signals from the interface unit (6), and, responsive to the control signals, configure the operation of components of the ASIC (3) and/or the ultrasound scanning unit (2) by setting operation parameters, wherein the controller (33) is adapted to change operation parameters of the ASIC (3) and/or the ultrasound scanning unit (2) during an insonification cycle and/or from one insonification cycle to the next.
Claims
1. A digitizing ASIC for an ultrasound scanning unit of an ultrasound system, comprising: an array of analogue-to-digital converters adapted to receive ultrasound signals acquired with an ultrasound transducer array and to convert the ultrasound signals into digitized ultrasound data; a memory module operably coupled to the array of analogue-to-digital converters and adapted to store the digitized ultrasound data; a transmitter, in particular a transceiver, operably coupled to the memory module and adapted to transfer the digitized ultrasound data stored in the memory module to a remote interface unit; and a controller adapted to receive control signals from the interface unit, and, responsive to the control signals, configure the operation of components of the ASIC and/or the ultrasound scanning unit by setting operation parameters, wherein the controller is adapted to change operation parameters of the ASIC and/or the ultrasound scanning unit during an insonification cycle and/or from one insonification cycle to the next, wherein the controller is adapted to control a selection and/or processing of the digitized ultrasound data, in particular by an on-chip digital signal processor of the digitizing ASIC, wherein the selection and/or processing may include one or more of the following: compressing digitized ultrasound data; subsampling of digitized ultrasound data; combining digitized ultrasound data from several transducer elements and/or from different signal channels and/or from different insonification cycles, in particular by weighted summation; adjusting word width; adjusting digital gain; signal clipping; re-interpolation of digitized ultrasound data; and filtering and decimation.
2. The digitizing ASIC according to claim 1, wherein the controller is adapted to change one or more of the following operation parameters of the analogue-to-digital converters during an insonification cycle and/or from one insonification cycle to the next: the operation parameters of an analogue front-end of the analogue-to-digital converters, in particular an amplification and/or a filter of the analogue input signals; a selection of active analogue-to-digital converters; a sampling frequency; a resolution of the analogue-to-digital converters, in particular a variation in resolution during one insonification cycle; and acquisition delays and/or and acquisition duration.
3. The digitizing ASIC according to claim 1, wherein the controller is adapted to, responsive to the control signals, select portions of the digitized ultrasound data to be stored in the memory module, and/or to select portions of the digitized ultrasound data stored in the memory module that are to be transferred to the interface unit.
4. The digitizing ASIC according to claim 1, wherein the digitizing ASIC is adapted to consecutively apply multiple control modes, each control mode, comprising a set of operation parameters, wherein the next control mode, is activated after a predetermined time or due to detection of an internal or external trigger event.
5. The digitizing ASIC according to claim 1, wherein the digitizing ASIC comprises at least two register banks, wherein the at least two register banks are configured to store operation parameters, wherein the digitizing ASIC is adapted to apply operation parameters of a first register bank, in particular during a current insonification cycle, while overwriting operation parameters of a second register bank for use during a next insonification cycle and/or data processing.
6. The digitizing ASIC according to claim 1, wherein the memory module and the array of analogue-to-digital converters are clocked from a first clock domain and the transceiver is clocked from a second clock domain.
7. An ultrasound scanning unit, comprising an ultrasound transducer array and a digitizing ASIC according to claim 1.
8. The ultrasound scanning unit according to claim 7, wherein components of the ultrasound scanning unit, in particular the ultrasound transducer array, an analogue ASIC and/or the digitizing ASIC, are arranged on a wearable patch.
9. A master controller for an interface unit of an ultrasound system, the interface unit being adapted to be coupled to at least one ultrasound scanning unit, in particular to at least one ultrasound scanning unit according to claim 7, wherein the master controller comprises or is part of a data processing unit and is configured to dynamically generate and send control signals for controlling a digitizing ASIC of the at least one ultrasound scanning unit based on digitized ultrasound data received from said at least one ultrasound scanning unit by the interface unit and/or data produced by the interface unit from digitized ultrasound data received from said at least one ultrasound scanning unit.
10. The master controller according to claim 9, wherein the master controller is configured to generate and send control signals related to a next insonification cycle during data collection of a current insonification cycle.
11.-14. (canceled)
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0117] The invention shall now be illustrated by means of embodiments with reference to the attached drawings, in which:
[0118] FIG. 1 shows a schematic representation of an ultrasound system according to an embodiment of the invention;
[0119] FIG. 2 shows a block diagram of a digitizing ASIC according to an embodiment of the invention;
[0120] FIG. 3 shows the concept of the connection between digitizing ASIC and master controller according to an embodiment of the invention;
[0121] FIG. 4 shows an exemplary timing diagram on data acquisition;
[0122] FIG. 5 shows a flow chart of the application of the invention's principle according to an exemplary embodiment;
[0123] FIG. 6 shows a representation of operational parameters in a register bank with different control modes according to an embodiment of the invention;
[0124] FIG. 7 shows a sketch of an ultrasound system according to an embodiment of the invention;
[0125] FIG. 8 shows a schematic example of four data blocks of an array of data blocks with 44 samples each;
[0126] FIG. 9 shows the schematic structure of a decoding unit in the form of an upsampling network according to an embodiment of the invention;
[0127] FIG. 10 shows the schematic structure of a decoding unit in the form of a u-net based network according to an embodiment of the invention;
[0128] FIG. 11 shows a comparison of original and reconstructed 2D RF data plots;
[0129] FIG. 12 shows different subsampling patterns with varying subsampling rate;
[0130] FIG. 13 shows two different subsampling patterns and an interleaved subsampling pattern, wherein the two different subsampling patterns are combined;
[0131] FIG. 14 shows the concept of a decoding unit using two interleaved subsampling patterns from consecutive US frames;
[0132] FIG. 15 shows the working principle of a decoding unit comprising a first decoder and a second decoder.
DETAILED DESCRIPTION OF EMBODIMENTS
[0133] Throughout the FIGS., the same or corresponding features/elements of the various embodiments are designated with the same reference numbers.
[0134] FIG. 1 shows a schematic representation of an ultrasound system 1 according to an embodiment of the invention. In this embodiment, the system comprises a computer 12, an interface unit 6 with a master controller 7, and three ultrasound scanning units 2 that are connected to the interface unit 6. The ultrasound scanning units 2 each comprise an analogue ASIC 8 with a transducer array 21. The transducer array 21 may be an integral part of the analogue ASIC 8 or one or multiple transducer arrays 21 may be connected to the analogue ASIC 8. For example, the transducer arrays 21 may be a matrix transducer array 21 which may be monolithically arranged on top of the analogue ASIC 8. The analogue ASIC 8 is connected to a digitizing ASIC 3 of the ultrasound scanning unit 2. The connection between the ASICS 8, 3 may for example be a side-by-side connection or a stacked arrangement using through silicon vias (TSV). The analogue and digital ASIC function may also be combined in a single mixed-signal ASIC.
[0135] The digitizing ASIC 3 comprises a controller 33 that is adapted to control various operation parameters of the ultrasound scanning unit 2, and in particular of the digitizing ASIC 3. Furthermore, the digitizing ASIC 3 comprises an array of ADCs 31 that are configured to digitize RF ultrasound signals from the analogue ASIC 8. The digitized ultrasound data may then be stored in a memory module 32 of the digitizing ASIC 3, e.g. an on-chip memory such as SRAM or DRAM, and possibly processed. According to instructions of the controller 33, the data may be filtered, selected and/or processed, in particular in real time, prior to storing it in the memory module 32 or after storing it in the memory module 32. Accordingly, the data may be compressed on the digitizing ASIC 3, which reduces the amount of data that needs to be transferred to the interface unit 6. In order to transfer the selected and processed digitized ultrasound data, there is a serial link 11, which may in particular consist of multiple data lanes, between each ultrasound scanning unit 2 and the interface unit 6. The digitized ultrasound data may be encoded and/or serialized via an encoder/serializer 43 and transmitted by a transceiver 44. The digitizing ASIC 3 may also comprise an encoding unit 343 that is configured to subsample the data. The encoding unit 343 may be configured to subsample data after it is stored in the memory module 32. However, the encoding unit 343 may also be configured to subsample the data prior to storing it into the memory module 32 and possibly even prior to digitizing it with the array of ADCs 31. Optionally, the ultrasound scanning unit 2 may comprise a VCSEL 24 to enable an optical connection, e.g. via fiber optics, to the interface unit 6. Alternatively the digitized ultrasound data may also be transferred electrically, e.g. via UTP. The connection between the analogue ASIC 8 and the digitizing ASIC 3 may be designed such that it is possible to use multiple digitizing ASICs 3 with one analogue ASIC 3 or vice versa. Furthermore, the ultrasound scanning unit 2 shown in FIG. 1 comprises an EPROM 25 to store a calibration and/or specifications about the ultrasound scanning unit 2.
[0136] The interface unit 6 comprises a data processing unit 61 including a beamformer 62 and the master controller 7. The master controller 7 is adapted to send control signals to the controller 33 depending on data analysis performed by the data processing unit 61. After beamforming and maybe additionally processing the data on the interface unit, the data is transferred to the computer 12 that comprises a user interface 13. In this case, in order to be independently operable, the interface unit 6 comprises a battery 63 and a power management unit (PMU) 64. Therefore the connection between the interface unit 6 and the computer 12 may be wireless.
[0137] FIG. 2 shows a block diagram of a digitizing ASIC 3 according to an embodiment of the invention. The digitizing ASIC comprises an array of ADCs 31, wherein each ADC 39 is combined with an anti-aliasing filter 38, an amplifier 37 and optionally with a single-to-differential converter (not shown). The anti-aliasing filter 38, the amplifier 37 and a single-to-differential converter may be part of the analogue front-end of the digitizing ASIC 3. In this schematic example, the array 31 consists of four ADCs 39, however in reality there may be more ADCs 39, e.g. 32-256 ADCs 39. In particular, there may be one ADC 39 for each signal channel of the analogue ASIC. Alternatively, two or more signal lines/channels may share one ADC 39, as described above. Prior to storing the data into the memory module 32, the data may be subjected to digital data selection 53, filtering, e.g., over time or channel, and/or processing 54 as controlled by the controller 33. The processing may change during the acquisition, e.g. from one insonification cycle to the next, and may be application specific. For example, the resolution or the ADC sampling frequency may change during a single acquisition to perform subsampling, resolution scaling, etc. as part of an intelligent data reduction algorithm. The controller 33 may also control the analogue front-end modules, e.g., the amplifier 37 and the anti-aliasing filter 38. These modules may for example be biased only when actively acquiring relevant echo signals and turned off, i.e., bias currents are turned off, otherwise. In this embodiment the memory module 32 consists of multiple memory units 41, in particular one for each ADC 39. Preferably the capacity of the memory is sufficient to store the RF-data related to one or few insonification events, e.g., the memory capacity may be in the order of 30-100 kbits per signal channel. After storing the relevant data on the memory module 32, additional data selection 53, filtering, and processing 54, e.g., normalizing and/or clipping, may be instructed by the controller 33 prior to being forwarded to the interface unit 6 via a transmitter 44 or transceiver and a serial link 11 which may comprise multiple lanes in serial data format. Hence, the controller 33 is adapted to control operation parameters of the array of ADCs 31, the memory module 32 and/or the transmitter/transceiver 44. Furthermore, the data may be encoded, e.g., with an 8B/10B encoding, and/or serialized by the encoder and serializer 44. The memory module 32 and the array of ADCs 31 are clocked from a first clock domain 45 while the encoder and serializer 43 are clocked from a second clock domain 46. An elastic buffer 42 serves as a handshake mechanism between the two clock domains and guarantees data integrity. The decoupling into two clock domains, i.e., the decoupling of the data acquisition bandwidth and the serial data link bandwidth is possible due to the memory module 32. This allows to compensate for the often much faster data acquisition bandwidth compared to the slower serial data link speed 11 which is often fixed, while the data acquisition bandwidth (ADC sampling frequency, resolution, number of active channels) may vary, even during an insonification cycle. While data acquisition should be synchronized with the insonification event, data transfer to the interface unit 6 can continue on afterwards until all selected data has been transferred. The two clock domains 45, 46 are generated a lower frequency reference clock via a PLL 47 (phase locked loop).
[0138] FIG. 3 shows the concept of the connection between digitizing ASIC and master controller according to an embodiment of the invention. Based on control signals 71 from the master controller 7, the controller 33 is adapted to change operation parameters of the digitizing ASIC 3. In this embodiment, the operation parameters comprise an analogue signal selection 51, e.g., filtering of analogue US signals, digitization 52, e.g., setting gain, ADC sampling frequency, bandwidth and/or resolution, digital data selection 53, e.g., an additional subsampling of data stored in the memory module 32, and filtering & processing 54, e.g., setting word width, digital gain, applying a function, etc. The filtered and processed data are then forwarded to the interface unit 6 and analyzed by the master controller 7 for determining the next control signals 71.
[0139] FIG. 4 shows an exemplary timing diagram for data acquisition. The digitizing ASIC 3 is pre-programmed using acquisition delays (e.g., T1A_B1, T2A_B1) and acquisition duration parameters (e.g., T1B_B1, T2B_B1), wherein the second character indicates the signal channel or transducer element (1 . . . x). The timing parameters related to the data acquisition of the next insonification event may be programmed during the data acquisition of the current event. This is made possible by using bank-switchable register tables (denoted by indexes B1 and B2). After a trigger event 91, e.g., from the master controller, a predetermined acquisition delay 92 is applied, e.g., to wait for the ultrasound waves after an insonification to be received by the transducer and transmitted to the digitizing ASIC 3. After the delay 92 follows data acquisition during an acquisition duration 93. Different signal channels, i.e., different ADCs 39 (ADC1, . . . ADCX) of the array of ADCs 31 may have different timing parameters. In this example a first channel 95 (ADC1) and a further channel 96 (ADCX) is shown representative of all channels. On the left side of the diagram, the timings of a first bank 97 are shown (also indicated by the index B1 in the timing parameters). After another trigger event 91 is received the timings of a second bank 98 are applied (indicated by the index B2 in the timing parameters). On the lower part of the diagram is shown the stream of serial data 94 that is forwarded to the interface unit 6. Here, a higher line denotes a (maximum) stream of serial data 94, while the lower line denotes a pause of the data stream. As can be seen, the serial stream goes on even after the end of the acquisition duration 93 of the signal channels (during dead time). Hence, the data is temporarily stored in the memory module 32, in order to compensate for the slower speed of the serial link 11 to the interface unit 6 compared to the acquisition and processing speed. The next acquisition is timed such that the acquisition duration of the second bank 98 sets in only after the serial data 94 corresponding to the first bank 97 has been completely forwarded. In this case, there is a small pause in in the stream of serial data 94 after completion of forwarding data acquired with the operation parameters of the first bank 97 before the data acquired with the operation parameters of the second bank 98 are streamed out. This principle may be continued throughout several banks.
[0140] FIG. 5 shows a flow chart of the application of the invention's principle according to an exemplary embodiment. The first control mode 101, which sets operation parameters 110 as instructed by the control signals from the master controller, is initiated by an external trigger event 91 (not shown). This trigger event is synchronized withbut not necessarily equal tothe insonification event. The first mode 101 may for example be used as a hold-off mode. There may be no new data acquisition and the ADCs 39 may not execute analogue-to-digital conversions. Old data available in the memory module 32 may be transferred to the master controller 7. A comma code may be inserted in the encoded data to guarantee word synchronization. After a predetermined time, monitored by a time out monitoring 111, the second control mode 102 is activated to set operation parameters 110. It may be the first data acquisition mode, ADCs 39 may be provided with the selected sampling clock and digitizing of analogue input ultrasound signals starts. Analogue input gains may be relatively low and ADC resolution may be high. Digital word widths may be high as well. Again, after the monitored timing 111, the next control mode, i.e. the third control mode 103 is activated to set new operation parameters 110. The third control mode 103 may be the second data acquisition mode, analogue input settings may be adapted, the number of active ADC channels may be adapted as well. Typically, the amplitude and the dynamic range of the analogue signals tend to reduce over time, which may be matched. However, earlier system learnings, evaluated by the master controller may have shown otherwise and, hence an individual and dynamic adjustment may be applied. Operation parameters may also be adjusted to observe special regions of interest. After the timing out 111 of the third control mode 103, the operation parameters of the fourth control mode 104 may be set 110. The fourth control mode 104 may be the last data acquisition mode, analogue front-end settings may be optimized for now weaker signals, the dynamic signal range may be low as well as the signal amplitude, which may be compensated/matched for. If, for example, signal noise is dominated by the analogue front-end 48, it may not make sense to increase the analogue gain further. However, it may make sense to transfer relevant data bits only, e.g. the LSBs (least significant bits) of the signal. Furthermore, the sampling frequency may be reduced in order to match the lower-frequency signal bandwidth. After the timing out 111 of the fourth control mode 104, the fifth control mode 105 may be used to transfer pre-collected data while the analogue front-end 48 and ADCs 39 may be turned off. The digitizing ASIC 8 may stay in this mode until the acquired data has been transferred to the master controller 7. A trigger event 91, e.g., sent from the master controller 7 to the controller and recognized by a trigger reception monitoring 112, may then end the fifth control mode 105 and initiate a switching of the register banks. Hence, after detection of the trigger 91, the selected register bank may be swapped and the analogue ASIC 8 and digitizing ASIC 3 functions may be prepared for new data acquisition. Next to that, read- and/or write-pointers of the memory module 32 may be reset and a special comma code may be asserted in the 8B/10B-encoded data.
[0141] FIG. 6 shows a representation of operational parameters in a register bank with different control modes according to an embodiment of the invention. In this embodiment, the analogue front-end 48 of the digitizing ASIC 3 is programmed with a set of analogue parameters, i.e,. gain 121, analogue filter settings, in this case bandwidth 122, circuit slew rate characteristics, circuit bias currents 123. The programming of the ADCs 39 includes selection of active ADC channels 124, i.e. enabling of ADCs 124, control of ADC resolution 125 and optionally sampling frequency. Unused ADC channels, including the pertaining analogue front-end 48, are switched off to minimize power dissipation. The digital processing 115 functions are programmed to select specific data, choose specific word widths 126, apply digital gain 127, and to execute specific data processing, e.g. signal clipping 129, signal filtering and/or the enabling of functions 128, e.g. signal summation. The mode-related programming parameters also indicate the duration 116 (expressed in clock cycles), during which the ASIC will remain in a particular mode of operation and the timing 130 of the individual delays and acquisitions. The duration 116 is controlled via timing in the case of the first 101, second 102, third 103, and fourth 104 control mode. The ASIC will switch to the next control mode when the pertaining counter times out. The fifth mode 105 is an exception. The digitizing ASIC 3 will stay in the fifth mode 105 until an external trigger 91 or reset has been received. At that moment, the digitizing ASIC is configured to switch back to the first mode 101, possibly of another register bank, and the mode cycle repeats.
[0142] FIG. 7 shows a sketch of an ultrasound system 1 according to an embodiment of the invention. The ultrasound system 1 comprises the ultrasound scanning unit 2, the interface unit 6 with the master controller 7, a beamformer 62 and a battery 63. The ultrasound system 1 further comprises a computer with a user input device 16 and a screen configured to show visualized data 14.
[0143] FIG. 8 shows a schematic example of four data blocks 302 which are repeated of the complete RF signal data array 301 with 44 samples 303 each. The data blocks 302 comprise an x-axis for the transducer elements, i.e. in this case there are four transducer elements in each data block, and a t-axis for the fast time, i.e. in this example there are four discrete time values in each data block 303. Each data block 302 has been subsampled by an encoding unit 43 by selecting specific samples 304 to keep. The circles denote kept samples 304, while the empty boxes denote omitted samples 305. Thus, the subsampling rate in this case is , equally for all data blocks, i.e. every second sample is kept. Furthermore one subsampling pattern is repeated for all data blocks, whereby a uniform sampling density is obtained. This allows for efficient processing by a trained algorithm, e.g., a fully convolutional neural network, as the subsampling pattern is the same for each data block, hence it is block shift invariant
[0144] FIG. 9 shows the schematic structure of a decoding unit 310 in the form of an upsampling network according to an embodiment of the invention. The upsampling network, which is in particular a neural network, consists of convolutional layers 307 and upsampling layers 308. As can be seen, in this embodiment there are alternatingly arranged three convolutional layers 307 and one upsampling layer 308, wherein the data is upsampled a total of three times. Hence, for example data of an US frame that has been subsampled with a subsampling rate of may be input and upsampled to the original frame size, i.e., a rate of 1/1. This may in particular be carried out gradually, in particular in factors of 2, wherein the samples in the x-axis and t-axis are increased at each upsampling layer, i.e., from [Elements/4, Time/4] to [Elements/2, Time/2] to [Elements/1, Time/1], the latter being the original size.
[0145] FIG. 10 shows the schematic structure of a decoding unit 310 in the form of a u-net based neural network (NN) according to an embodiment of the invention. It comprises a downsampling (encoder) part, wherein three convolutional layers 307 and one downsampling layer 309 are alternatingly arranged, respectively, such that input data is downsampled three times in total. Furthermore it comprises an upsampling (decoder) part following the downsampling part, wherein three convolutional layers 307 and one upsampling layer 309 are alternatingly arranged, respectively, such that input data is upsampled three times in total. As input, a tensor with the original (and thus also final size) may be used, wherein the value zero is set at non-sampled locations while the sampled values are set at their corresponding location. The decoding unit 310 of this embodiment is trained to re-interpolate the missing, i.e. set to zero, values. Furthermore, the neural network of the decoding unit 310 comprises skip connections 319, i.e., connections that work as shortcuts and allow to jump over some layers in between. Skip connections 319 may be useful when training the neural network for alleviating the problem of vanishing gradients. By training the NN decoding unit 310 of FIG. 9 or 10 together with the encoding unit 343 to reconstruct the full RF signal data, as used as input, e.g., like training an auto-encoder, one can obtain the weights of the decoding unit. In particular, for each chosen subsampling pattern, one may retrain and obtain different weights.
[0146] Experiments have been carried out with a Verasonics setup in combination with an L7-4 probe. The Verasonics setup captures the raw RF signal, and sample rate converts the RF data to 4 samples per sound wavelength. The ultrasound probe was typically used in a synthetic aperture mode, i.e. getting the RF signal from all transducer elements, e.g. like in plane wave imaging/multi-angled plane wave. In both cases, beamforming and additional signal/image processing were done on a dedicated data processing unit, such as a DSP, GPU or AI accelerator, and may be realized in leading-edge low-voltage MOSfet technology using ultimate small feature size (e.g., FinFET technology, 5 nm). Typically, analogue frontend ASICs (ASIC technology equipped with high-voltage transistors and low voltage MOS transistors with large feature size, e.g. 180 nm) and/or digital frontend/digitizing ASICs (mature low voltage ASIC technologies e.g. 40 nm) may be used. Thus the amount of compute available in the high-end gpu or the embedded gpu of the data processing unit typically is much higher due to more advanced CMOS process technology.
[0147] FIG. 11 shows a comparison of original and reconstructed 2D RF data plots, wherein only 1 in 4 pixels is sampled. Here, the horizontal axis shows the fast time axis, corresponding to the depth of the image, and the vertical axis shows the 128 elements of the transducer. The original RF data plot is shown in the upper image and the reconstruction is shown in the middle image. The difference between original and reconstructed data, i.e. the error, is shown in the bottom image. As can be seen from the bottom image, there are some visible errors, i.e. the signal is not zero. Hence, the method is a lossy compression method.
[0148] A closer analysis has shown that the error is higher in regions with a high signal amplitude. However, when subsampling by a factor of 4, reconstructing the RF signal with a neural network, and beamforming to obtain the resulting images and looking at the artefacts this lossy compression creates, one has found that the differences in the beamformed image are so small, that the difference can hardly be seen by the human eye. Calculating the difference has shown that the difference between original and reconstructed image is indeed quite random and only weakly correlated with the image content.
[0149] It has turned out that the reconstruction quality is directly proportional to the number of sampling points. Hence, the lower the data rate, the higher the distortion. A resulting peak signal-to-noise-ratio (PSRN) in the beamformed image decreases when sampling with less points. In order to compare the reconstruction results with a well-known interpolation method, the grid data interpolation method from scipy (https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.griddata.html) was applied. This method implements bicubic interpolation on a non-regular grid. The subsampling has been implemented on top of an IQ encoded RF-data. When subsampling with a factor of 4, i.e., a subsampling rate of , a 6 dB drop in PSNR has been observed when using bicubic interpolation (20.4 dB) compared to using a neural network based interpolation as described above. (26.5 dB). Thus, the NN based decoding has performed considerably better than bicubic interpolation.
[0150] FIG. 12 shows different subsampling patterns with varying subsampling rate, i.e., R=, and , in one data block 302. A subsampling rate of R= yielded a PSNR of the beamformed b-mode image of 42 dB, a rate of R= yielded a PSNR of 26.5 dB and a rate of R= yielded a PSNR of 20 dB.
[0151] When the scene is static or slowly moving, one can use this knowledge (i.e. that one frame will have significant similarity to the previous and subsequent frame), by sampling with slightly different patterns, i.e. phase shifted and/or interleaved patterns. FIG. 13 shows two different subsampling patterns (drawing on the left and middle) and an interleaved subsampling pattern (drawing on the right side) within one data block 302, wherein the two different subsampling patterns are combined. In the left drawing samples are kept according to a first pattern 304 and in the middle drawing samples are kept according to a second pattern 314. Both patterns have a subsampling rate of . The first pattern 304 and the second pattern 314 taken for consecutive image frames are combined on the right drawing obtaining an interleaved pattern with an apparent subsampling rate of . As described with respect to FIG. 12, a higher subsampling rate such as R=, and thus possibly also this combined pattern, may result in a much better PSNR than a lower subsampling rate such as R=, i.e., the individual patterns. This knowledge can be used to build a more advanced reconstruction algorithm as for example shown in FIG. 14. Starting from 6 consecutive original or full frames 311 (F.sub.2, F.sub.1, F.sub.+0, F.sub.+1, F.sub.+2 and F.sub.+3), the encoding unit 343 applies alternating subsampling patterns in an array of data blocks 301 as shown on the left side of FIG. 14. Next, 6 of these subsampled frames 312 (F.sub.2A, F.sub.1.sup.B, F.sub.+0.sup.A, F.sub.+1.sup.B, F.sub.+2.sup.A and F.sub.+3.sup.B) are used as input to the neural network, e.g. as additional channels. The numbers A and B denote the first and second subsampling pattern, respectively (see drawing on the left of FIG. 14). The decoding unit 310, i.e., the neural network, is trained to reconstruct the central two frames as reconstructed frames 313 (F.sub.+0 and F.sub.+1) from the subsampled frames 312. In this embodiment, the NN may effectively learn to reconstruct the full RF signal or frames, but also to predict motion and use this to further improve the reconstruction, i.e., by learning how to deal with motion in the scene. The decoding unit 310 is trained such, that it may shift this procedure two frames, i.e. in this case to (F.sub.+0, F.sub.+1, F.sub.+2, F.sub.+3, F.sub.+4, F.sub.+5), and apply it again, i.e., with output (F.sub.+2 and F.sub.+3), and continue doing so, effectively enabling a stream based processing. It has turned out that the PSNR of an image created from interleaved patterns such as the one on the right side of FIG. 13, i.e., with an apparent subsampling rate of originating from two individual patterns with a rate of , is between 35,8 and 40.4 dB. This is a significant improvement over the individual PSNR of patterns with a subsampling rate of (e.g. middle drawing of FIG. 12 and left and middle drawings of FIG. 13), while being only slightly worse than the 42 dB from a real subsampling rate of (e.g. as shown in the left drawing of FIG. 12).
[0152] FIG. 15 shows the working principle of an alternative decoding unit 310 comprising a first decoder 315 and a second decoder 316. Both, the first and the second decoder may be or comprise neural networks. The first decoder 315 is configured to calculate latent representations 317 (Z.sub.1, Z.sub.0 and Z.sub.+1) from the subsampled frames 312. In particular, in this embodiment, the first decoder 315 is configured to create a first latent representation 317 (Z.sub.1) from the first two subsampled frames 312 (F.sub.2A and F.sub.1.sup.B), wherein the subsampled frames 312 have different, in particular complementary, subsampling patterns, i.e. pattern A and B, respectively. Analogously, a second latent representation (Z.sub.0) is created from the third and fourth subsampled frames (F.sub.0.sup.A and F.sub.+1.sup.B) and a third latent representation (Z.sub.+1) is created from the fifth and sixth subsampled frames (F.sub.+2.sup.A and F.sub.+3.sup.B). The latent representations 317 are then combined 318 (e.g., concatenated) and decoded by the second decoder 316 in order to output reconstructed frames 313 (F.sub.+0.sup.A and F.sub.+3.sup.B). By shifting the frames by two subsampled frames and one latent frame, i.e. to (Z.sub.0, Z.sub.+1 and Z.sub.+2), the decoding unit 310 may reuse two of the latent frames, i.e. Z.sub.0 and Z.sub.+1, and only compute Z.sub.+2 via the first decoder 315 in order to create the next two reconstructed frames 313 (F.sub.+2 and F.sub.+3) via the second decoder 316. This scheme may be continued in order to create a series of reconstructed frames 313. By only having to calculate the latent representations 317 once but using them for multiple reconstructed frames 313, therefore not needing to fully process six input frames, i.e., subsampled frames 312, each time, less computing is required in total.
[0153] The above-discussion is intended to be merely illustrative of the present invention and should not be construed as limiting the appended claims to any particular embodiment or group of embodiments. Thus, while the present system has been described in particular detail with reference to exemplary embodiments, it should also be appreciated that numerous modifications and alternative embodiments may be devised by those having ordinary skill in the art without departing from the broader and intended spirit and scope of the present system as set forth in the claims that follow. Accordingly, the specification and drawings are to be regarded in an illustrative manner and are not intended to limit the scope of the appended claims.
LIST OF REFERENCE SIGNS
[0154] 1 ultrasound system [0155] 2 ultrasound scanning unit [0156] 3 digitizing ASIC [0157] 6 interface unit [0158] 7 master controller [0159] 8 analogue ASIC [0160] 11 serial link (lanes) [0161] 12 computer/system [0162] 13 user interface [0163] 14 visualized data [0164] screen [0165] 16 user input device [0166] 21 transducer array [0167] 24 VCSEL [0168] EPROM [0169] 31 array of analogue-to-digital converters [0170] 32 memory module [0171] 33 controller [0172] 36 transmitter [0173] 37 amplifier [0174] 38 anti-aliasing filter [0175] 39 analogue-to-digital-converter [0176] 41 memory unit [0177] 42 elastic buffer [0178] 43 encoder & serializer [0179] 44 transceiver/transmitter/buffer [0180] first clock domain [0181] 46 second clock domain [0182] 47 PLL [0183] 48 analogue front-end [0184] 51 analogue signal selection [0185] 52 digitization [0186] 53 digital data selection [0187] 54 filtering & processing [0188] 61 data processing unit of the interface unit [0189] 62 data processor/beamformer [0190] 63 battery [0191] 64 power management unit (PMU) [0192] 71 control signals [0193] 91 trigger (event) [0194] 92 acquisition delay [0195] 93 acquisition duration [0196] 94 serial data [0197] 95 first channel/ADC data collect [0198] 96 further channel/ADC data collect [0199] 97 first register bank [0200] 98 second register bank [0201] 101 first control mode [0202] 102 second control mode [0203] 103 third control mode [0204] 104 fourth control mode [0205] 105 fifth control mode [0206] 110 set operation parameters [0207] 111 time out monitoring [0208] 112 trigger reception monitoring [0209] 113 switching register bank [0210] 115 digital processing [0211] 116 duration [0212] 121 gain [0213] 122 bandwidth [0214] 123 bias [0215] 124 ADC enable [0216] 125 ADC resolution [0217] 126 word width [0218] 127 digital gain [0219] 128 function enable [0220] 129 clip [0221] 130 timing [0222] 301 array of data blocks [0223] 302 data block [0224] 303 sample [0225] 304 kept sample (first pattern) [0226] 305 omitted sample [0227] 307 convolutional layer [0228] 308 upsampling layer [0229] 309 downsampling layer [0230] 310 decoding unit [0231] 311 full frames [0232] 312 subsampled frames [0233] 313 reconstructed frames [0234] 314 kept sample (second pattern) [0235] 315 first decoder [0236] 316 second decoder [0237] 317 latent representations [0238] 318 combined latent representations [0239] 319 skip connection [0240] 343 encoding unit
