System and method for multipath processing of image signals

Abstract

The invention generally relates to intravascular imaging system and particularly to processing in multimodal systems. The invention provides an imaging system that splits incoming image data into two signals and performs the same processing step on each of the split signals. The system can then send the two signals down two processing pathways. Methods include receiving an analog image signal, transmitting the received signal to a processing system, splitting the signal to produce a first image signal and a second image signal, and performing a processing operation on the first image signal and the second image signal. The first and second signal include substantially the same information as one another.

Claims

1. An intravascular ultrasound (IVUS) imaging system, the system comprising: an elongated catheter configured to be positioned within a blood vessel of a patient, wherein the elongated catheter comprises a proximal portion, a distal portion, and an ultrasound transducer at the distal portion; and a processing circuit configured for communication with the elongated catheter, wherein the processing circuit is operable to: activate generation of an IVUS image with a first visual appearance; activate generation of the IVUS image with a second visual appearance; receive a single analog image signal from the ultrasound transducer; digitize, using an analog-to-digital converter, the single analog image signal to a digital image signal; split the digital image signal into a first image signal and a second image signal, the first image signal comprising a copy of the second image signal; perform a first set of processing operations on the first image signal along a first signal path, wherein the first set of processing operations includes a first visual appearance operation inconsistent with the generation of the IVUS image with the second visual appearance; perform a second set of processing operations on the second image signal along a second signal path, wherein the second set of processing operations includes a second visual appearance operation different from the first visual appearance operation; generate the IVUS image with the first visual appearance based on the first image signal and the IVUS image with the second visual appearance based on the second image signal; and output the IVUS image with the first visual appearance and the IVUS image with the second visual appearance to a display in communication with the processing circuit.

2. The IVUS imaging system of claim 1, wherein the first set of processing operations comprises generating qualitative or quantitative blood flow data, and wherein the IVUS image with the first visual appearance includes visual representations of the qualitative or quantitative blood flow data.

3. The IVUS imaging system of claim 1, wherein the first set of processing operations comprises automatically generating a boundary in the IVUS image with the first visual appearance, and wherein the IVUS image with the first visual appearance includes the boundary.

4. The IVUS imaging system of claim 1, wherein the first set of processing operations includes graphic equalization of the first image signal using a mixer.

5. The IVUS imaging system of claim 1, wherein the second visual appearance operation includes tissue characterization based on at least one of parametric analysis, spectral analysis, or a neural network.

6. The IVUS imaging system of claim 5, wherein the IVUS image with the second visual appearance comprises a virtual histology image.

7. The IVUS imaging system of claim 6, wherein the tissue characterization is configured to identify at least one of plaque, dead tissue, or healthy tissue.

8. The IVUS imaging system of claim 1, wherein the first set of processing operations includes processing by a hardware amplifier or filter.

9. The IVUS imaging system of claim 1, wherein the first set of processing operations includes at least one of a band pass filter, a low-pass filter, a multi-band low pass filter, or an analog gain.

10. The IVUS imaging system of claim 1, wherein the second set of processing operations includes at least one of a band pass filter or an analog gain.

11. The IVUS imaging system of claim 1, wherein the processing circuit is further operable to: receive a first user input activating the generation of the IVUS image with the first visual appearance; receive a second user input activating the generation of the IVUS image with the second visual appearance; perform the first set of processing operations based on the first user input; and perform the second set of processing operations based on the second user input.

12. The IVUS imaging system of claim 1, wherein the second set of processing operations omits the first visual appearance operation.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 shows an IVUS system according to certain embodiments of the invention.

(2) FIG. 2 shows a control station for an imaging system.

(3) FIG. 3 shows an control panel of a control station of an imaging system.

(4) FIG. 4 describes components of computer device of certain embodiments.

(5) FIG. 5 provides a schematic of processing circuits.

(6) FIG. 6 is a diagram of ultrasound signal processing in two pathways for images

(7) FIG. 7 shows a multipath signal processing scheme for ultrasound images.

(8) FIG. 8 shows a signal processing scheme for an image and for a parametric analysis.

(9) FIG. 9 shows a multipath signal processing scheme for flexibility.

DETAILED DESCRIPTION

(10) The invention provides an imaging system that splits incoming image data into two signals and performs the same processing step on each of the split signals.

(11) FIG. 1 shows a diagram of an exemplary IVUS system 101 according to certain embodiments of the invention. An operator uses control station 110 and optional navigational device 125 to operate catheter 112 via patient interface module (PIM) 105. At a distal tip of catheter 112 is an ultrasonic transducer 114. Computer device 120 works with PIM 105 to coordinate imaging operations. Imaging operations proceed by rotating an imaging mechanism via catheter 112 while transmitting a series of electrical impulses to transducer 114 which results in sonic impulses being sent into the patient's tissue. Backscatter from the ultrasonic impulses is received by transducer 114 and interpreted to provide an image on monitor 103. System 101 is operable for use during diagnostic ultrasound imaging of the peripheral and coronary vasculature of the patient. System 101 can be configured to automatically visualize boundary features, perform spectral analysis of vascular features, provide qualitative or quantitate blood flow data, or a combination thereof. Systems for IVUS suitable for use with the invention are discussed in U.S. Pat. No. 6,673,015; U.S. Pub. 2012/0265077; and U.S. RE40,608 E, the contents of which are incorporated by reference in their entirety for all purposes.

(12) Operation of system 101 employs a sterile, single use intravascular ultrasound imaging catheter 112. Catheter 112 is inserted into the coronary arteries and vessels of the peripheral vasculature under angiographic guidance. Catheters are described in U.S. Pat. Nos. 7,846,101; 5,771,895; 5,651,366; 5,176,141; U.S. Pub. 2012/0271170; U.S. Pub. 2012/0232400; U.S. Pub. 2012/0095340; U.S. Pub. 2009/0043191; U.S. Pub. 2004/0015065, the contents of which are incorporated by reference herein in their entirety for all purposes. System 101 may be integrated into existing and newly installed catheter laboratories (i.e., “cath labs” or “angiography suites”). The system configuration is flexible in order to fit into the existing catheter laboratory work flow and environment. For example, the system can include industry standard input/output interfaces for hardware such as navigation device 125, which can be a bedside mounted joystick. System 101 can include interfaces for one or more of an EKG system, exam room monitor, bedside rail mounted monitor, ceiling mounted exam room monitor, and server room computer hardware.

(13) System 101 connects to the IVUS catheter 112 via PIM 105, which may contain a type CF (intended for direct cardiac application) defibrillator proof isolation boundary. All other input/output interfaces within the patient environment may utilize both primary and secondary protective earth connections to limit enclosure leakage currents. The primary protective earth connection for controller 125 and control station 110 can be provided through the bedside rail mount. A secondary connection may be via a safety ground wire directly to the bedside protective earth system. Monitor 103 and an EKG interface can utilize the existing protective earth connections of the monitor and EKG system and a secondary protective earth connection from the bedside protective earth bus to the main chassis potential equalization post. Monitor 103 may be, for example, a standard SXGA (1280×1024) exam room monitor. System 101 includes computer device 120 to coordinate operations.

(14) Computer device 120 generally includes one or more processor coupled to a memory. Any suitable processor can be included such as, for example, a general-purpose microprocessor, an application-specific integrated circuit, a massively parallel processing array, a field-programmable gate array, others, or a combination thereof. In some embodiments, computer 120 can include a high performance dual Xeon based system using an operating system such as Windows XP professional. Computer 120 may be provided as a single device (e.g., a desktop, laptop, or rack-mounted unit, or computer 120 may include different machines coupled together (e.g., a Beowulf cluster, a network of servers, a server operating with a local client terminal, other arrangements, or a combination thereof).

(15) Computer device 120 may be configured to perform processing on more than one image modality (e.g., in parallel). For example, computer 120 may operate with real time intravascular ultrasound imaging while simultaneously running a tissue classification algorithm referred to as virtual histology (VH). The application software can include a DICOM3 compliant interface, a work list client interface, interfaces for connection to angiographic systems, or a combination thereof. Computer device 120 may be located in a separate control room, the exam room, or in an equipment room and may be coupled to one or more of a custom control station, a second control station, a joystick controller, a PS2 keyboard with touchpad, a mouse, or any other computer control device.

(16) Computer device 120 may generally include one or more USB or similar interfaces for connecting peripheral equipment. Available USB devices for connection include the custom control stations, optional joystick 125, and a color printer. In some embodiments, computer 120 includes one or more of a USB 2.0 high speed interface, a 10/100/1000 baseT Ethernet network interface, AC power jack, PS2 jack, Potential Equalization Post, 1 GigE Ethernet interface, microphone and line jacks, VGA video, DVI video interface, PIM interface, ECG interface, other connections, or a combination thereof. As shown in FIG. 1, computer device 120 is generally linked to control station 110.

(17) FIG. 2 shows a control system 110 according to certain embodiments. Control station 110 may be provided by any suitable device, such as a computer terminal (e.g., on a kiosk). In some embodiments, control system 110 is a purpose built device with a custom form factor. A slide out keyboard is located on the bottom for manual text entry. Control station 110 may be designed for different installations options. The station can be placed directly on a desktop surface. With the optional bedside mounting kit, control station 110 can be affixed directly to the bedside rail. This mounting kit is slipped over the rail and fixed in place by tightening two hand screws. Control station 110 can include a standard four hole VESA mount on the underside to allow other mounting configurations. Control system 110 may provide a simple-to-use interface with frequently-operated functions mapped to unique switches. Control station 110 may be powered from, and may communicate with, computer 120 using a standard USB 1.1 interface. The system may include a control panel 115. In some embodiments, multiple control panels 115 are mounted in both the exam room and/or the control room. A control station for use with the invention is discussed in U.S. Pat. No. 8,289,284, the contents of which are incorporated by reference in their entirety for all purposes.

(18) FIG. 3 shows an control panel 115 of control system 110 according to certain embodiments. Frequently-operated functions are mapped to contact closure switches. Those dome switches are covered with a membrane overlay. The use of dome switches provides a tactile feedback to the operator upon closure. Control panel 115 may include a pointing device such as a trackball to navigate a pointer on the graphical user interface of the system.

(19) Control panel 115 may include several screen selection keys. The settings key is used to change system settings like date and time and also permits setting and editing default configurations. The display key may be used to provide enlarged view for printing. In some embodiments, the print key prints a 6×4 inch photo of the current image on the screen. Control panel 115 may include a ringdown key that toggles the operation of ringdown subtraction. A chroma key can turn blood flow operations on and off. The VH key can operate the virtual histology engine. A record, stop, play, and save frame key are included for video operation. Typically, the home key will operate to display the live image. A menu key provides access to measurement options such as diameter, length, and borders. Bookmark can be used while recording a loop to select specific areas of interest. Select (+) and Menu (−) keys are used to make selections.

(20) In some embodiments, the system includes a joystick for navigational device 125. The joystick may be a sealed off-the-shelf USB pointing device used to move the cursor on the graphical user interface from the bedside. System 101 may include a control room monitor, e.g., an off-the-shelf 19″ flat panel monitor with a native pixel resolution of 1280×1024 to accept DVI-D, DVI-I and VGA video inputs.

(21) Control station 110 is operably coupled to PIM 115, from which catheter 112 extends. Catheter 112 includes an ultrasound transducer 114 located at the tip. Any suitable IVUS transducer may be used. For example, in some embodiments, transducer 114 is driven as a synthetic aperture imaging element. Imaging transducer 114 may be approximately 1 mm in diameter and 2.5 mm in length. In certain embodiments, transducer 114 includes a piezoelectric component such as, for example, lead zirconium nitrate or PZT ceramic. The transducer may be provided as an array of elements (e.g., 64), for example, bonded to a Kapton flexible circuit board providing one or more integrated circuits. This printed circuit assembly may rolled around a central metal tube, back filled with an acoustic backing material and bonded to the tip of catheter 114. In some embodiments, signals are passed to the system via a plurality of wires (e.g., 7) that run the full length of catheter 112. The wires are bonded to the transducer flex circuit at one end and to a mating connector in PIM 105 at the other. The PIM connector may also contains a configuration EPROM. The EPROM may contain the catheter's model and serial numbers and the calibration coefficients which are used by the system. The PIM 105 provides the patient electrical isolation, the beam steering, and the RF amplification. PIM 105 may additionally include a local microcontroller to monitor the performance of the system and reset the PIM to a known safe state in the event of loss of communication or system failure. PIM 105 may communicate with computer device 120 via a low speed RS232 serial link.

(22) FIG. 4 describes components of computer device 120 according to certain embodiments. Computer device 120 may include a motherboard 129 that includes an IVUS signal generation and processing system. FIG. 4 provides a high-level diagram and any box shown therein may be taken to represent a unit of hardware, a unit of functionality to be performed by one or more pieces of hardware, modules of software, or combination thereof. The signal generation and processing system may include an analog printed circuit assembly (PCA) 131, an digital PCA 133, one or more filter modules, and a VH board 135. Analog PCA 131 and digital PCA 133 are used to excite transducer 114 via catheter 112 and to receive and process the gray scale IVUS signals. The VH board 135 is used to capture and pre-process the IVUS RF signals and transfer them to the main VH processing algorithm as run by a computer processor system (e.g., dual Xeon processors). PIM 105 is directly connected to the analog PCA 131. A computer system that includes a computer, such as one like that depicted in FIG. 4, can be configured to perform the signal processing of the invention. Exemplary signal processing and systems therefore are discussed in U.S. Pat. Nos. 8,298,147; 8,187,191; 6,450,964; 5,485,845; U.S. Pub. 2012/0220874; U.S. Pub. 2012/0184853; and U.S. Pub. 2007/0232933, the contents of which are incorporated by reference herein in their entirety.

(23) FIG. 5 provides a schematic of analog PCA 131 and digital PCA 133 according to certain embodiments of the invention. Analog PCA 131 is shown to include amplifier 141, band pass filter 145, mixer 149, low pass filter 153, and analog-to-digital converter (ADC) 157. Here, the incoming signal is split just after bandpass filtering and prior to mixing by mixer 149. Mixer 149 performs the same function on each of two or more copies of the same signal. Then, low pass filter 153 performs the same low pass filtering function on each of the two or more copies of the same signal. Finally, ADC 157 converts each of the two or more copies of the same signal into a digital signal, and each of the two or more copies of the same digital signal is sent to the acquisition FPGA 165 for processing. Analog board 131 further includes an interface module 161 for PIM 105, as well as a clock device 169.

(24) The invention provides systems and methods that incorporate the insight that unexpected benefits can be provided by splitting the signal and performing the counter-intuitive process of performing the same signal processing operations on each copy of the split signal. Established thinking suggests splitting a signal to perform different operations on each copy of the split signal. Here, since computer 120 splits the pathway and performs the same processing operations on each pathway, the system is readily adaptable for future changes that would require different processing operations. In one particular example, a signal is processed according to a parametric processing operation that is used to characterize the imaged tissue, such as in virtual histology. In some embodiment, virtual histology applications involve a processing step that makes use of a neural network comprising interconnecting artificial neurons, e.g., as supplied by a computer processing system. In virtual histology, a neural network can be taught to characterize tissue and to infer tissue characteristics from an incoming set of data. In some embodiments, to be effective, a tissue analyzing neural network should be trained on signals that are processed according to the same pathway as the subject signals upon which the neural network operates. Where the neural network is receiving a signal from, for example, an IVUS system that is also being use to provide an image (e.g., on a monitor), and change in the IVUS processing pathway would diminish the capabilities of the neural network. For example, if a physician using IVUS wanted to turn ringdown subtraction on and off, the different signals could produce sub-par results in the VH application. In some cases, a new insight in IVUS imaging may call for an upstream change in signal processing. For example, it may be desired to implement a new graphic equalization on IVUS signals using a mixer 149. If a single signal goes through mixer 149 and is split thereafter, then implementing the graphic equalization would require re-training the neural network—i.e., the existing neural network learning would no longer be useful. Where a system has been deployed with a split upstream of mixer 149, as depicted in FIG. 5, even though mixer 149 may perform the same function on both paths when the system is first installed, the system is readily adapted to the new graphic equalization function when implemented, without disrupting the neural network.

(25) This adaptability becomes particularly useful when the subsequent changes involve inherently tunable processing parameters such as graphic equalization. One advantage of graphic equalization (or other parallelized processing such as multiband low pass filtering) is that it allows field-deployed fine tuning. If the signal were not split upstream of the tunable processor, then the other analysis modality (e.g., VH; high-versus low-frequency IVUS; blood flow characterization; simple data storage; others; or a combination thereof) would be compromised. While depicted in FIG. 5 as splitting the signal after bandpass filtering, one insight of the invention is that there can be value in splitting the signal at any point and subsequently performing the same processing step on the split. For example, in some embodiments, a signal is split within a processor such as a field programmable gate array and the two signal copies are also processed within that processor, with the processor performing the same operations on each signal path.

(26) Digital PCA 133 is depicted as having an acquisition FPGA 165, as well as a focus FPGA 171, and a scan conversion FPGA 179. Focus FPGA 171 provides the synthetic aperture signal processing and scan conversion. In some embodiments, a single signal is sent into acquisition FPGA 165; split within FPGA 165; and processed in duplicate within FPGA 165. In some embodiments, a single signal is sent into acquisition FPGA 171; split within FPGA 171; and processed in duplicate within FPGA 171. In certain embodiments, two like signals are both focused (e.g., according to a synthetic aperture modality) within FPGA 171, having been split from a single incoming signal anywhere upstream from the focusing transistor hardware of FPGA 171. FPGA 179 provides the final scan conversion of the transducer vector data to Cartesian coordinates suitable for display via a standard computer graphics card on monitor 103. Digital board 133 further optionally includes a safety microcontroller 181, operable to shut down PIM 105 as a failsafe mechanism. Preferably, digital PCA 133 further includes a PCI interface chip 175. It will be appreciated that this provides but one exemplary illustrative embodiment and that one or skill in the art will recognize that variant and alternative arrangements may perform the functions described herein. Clock device 169 and acquisition FPGA 165 may operate in synchronization to control the transmission of acquisition sequences.

(27) FIG. 6 provides a diagram of an embodiment of multipath processing of image signals. An ultrasound (US) transducer 114 receives (Rx) the signal that was transmitted (Tx). Further preprocessing operations 131 after the ultrasound (US) transmit (Tx) and receive (Rx), can include any desired bandpass filtering, analog gain, channel matching, demodulation, other steps, or a combination thereof. The analog signal is split into a first image signal and a second image signal and an analog-digital converter (ADC) performs a processing operation on the first image signal and the second image signal. Additional digital pre-processing operations 137 can include averaging, scaling, and DC offset correction. It will be appreciated by one skilled in the art that processing operation steps discussed herein can be performed in order other than shown here. Imaging processing operations 143 can then be performed and can include scaling, focus, processing, time gain compensation (TGC), ringdown subtraction, any additional processing, log conversion, other steps, or a combination thereof. Finally, an image can be prepared by image preparation operations 149 including, for example, any desired processing (e.g., contrast adjustment) and scan conversion.

(28) Image processing operations can include any such operations known in the art or those discussed herein and need not be limited to, nor include all of, those listed in FIG. 6. In general, a bandpass filter is used to remove or attenuate certain (e.g., non-desired or non-intended) frequencies from the signal. Gain generally refers to adjusting a signal strength (e.g., with an amplifier) to a desired strength. Demodulation can include removing a carrier signal from a signal. Averaging can refer to average a series of A lines for improved signal-to-noise ratio (sometimes called accumulation). Scaling can refer to a linear transformation of the signal. DC offset correction can include removing or adding a constant amplitude to a signal to produce an AC signal with an average amplitude near or at zero. Focusing can include using incoming sounds from adjacent Tx events to improve an A line. Time gain compensation (TGC) refers to adjusting the signal to compensate for a non-uniform signal strength based on attenuation in the IVUS backscatter strength relating to an amount of time that the sound signal was passing through tissue. Ringdown refers to transient sounds from vibration of transducer that can be removed in ringdown subtraction. Scan conversion can include transforming a 3D data set in preparation for a 2D display and can proceed by a fast Fourier transformation. By scan conversion, a B mode image is prepared from a set of A lines. Signal processing steps are discussed in more detail in U.S. Pat. Nos. 8,289,284; 8,187,191; 6,254,543; 6,200,268; U.S. Pub. 2012/0220874; U.S. Pub. 2011/0087104; and U.S. Pub. 2010/0234736, the contents of which are incorporated by reference herein in their entirety.

(29) FIG. 7 shows an alternative embodiment in which a signal is split just after digitization at the end of preprocessing operations 131. Here, any of the additional digital pre-processing operations 137, imaging processing operations 143, image preparation operations 149, or a combination thereof are performed in duplicate on the first image signal and the second image signal, which contain substantially the same information as one another through at least one of the shared processing steps.

(30) FIG. 8 depicts an embodiment in which a signal is received and preprocessing operations 131 are performed, after which the signal is split, and the first and second copies of the split signal are digitized. Additional digital pre-processing operations 147 may be performed on each of the first and second signal. Further, the first signal undergoes all of the signal processing discussed above with respect to FIG. 6. Here, the second signal undergoes processing in preparation for a parametric analysis such as a VH tissue characterization. Any of imaging processing operations 143 may be performed on the second (VH) signal path, however any or most of them may not. For example, ringdown subtraction is not performed on the VH path. Here, final preparation operations 149 for VH characterization can include a processing step that involves analysis by a neural network, after which the image can be used to characterize the tissue.

(31) FIG. 9 depicts a multipath signal processing scheme in which a second signal path is routed to storage for later flexibility. Here, even where the second signal path does not directly contribute to producing an analytical result simultaneously with the scan converted image provided by the first signal path, the presence of a second signal path shunt in system 120 allows a later analytical modality to be added to system 120. Any amount of additional digital pre-processing operations 137 may be performed on the first and second signal path after it is split. Then the second signal path can be routed to storage and stored. Storage of the second set of signals can have benefit because, even if processing of the first signal path changes over time, the accumulated signals stored from the second path have all been processed according to the same steps. A VH algorithm could be introduced and the neural network could be trained on the stored data. A subsequent problem in the first path could be diagnosed by calibrating first path images against path two data to establish, at what point in time, some change had occurred.

(32) As used herein, the word “or” means “and or or”, sometimes seen or referred to as “and/or”, unless indicated otherwise.

INCORPORATION BY REFERENCE

(33) References and citations to other documents, such as patents, patent applications, patent publications, journals, books, papers, web contents, have been made throughout this disclosure. All such documents are hereby incorporated herein by reference in their entirety for all purposes.

EQUIVALENTS

(34) Various modifications of the invention and many further embodiments thereof, in addition to those shown and described herein, will become apparent to those skilled in the art from the full contents of this document, including references to the scientific and patent literature cited herein. The subject matter herein contains important information, exemplification and guidance that can be adapted to the practice of this invention in its various embodiments and equivalents thereof.