Information processing apparatus, information processing method, and cell analysis system
10891734 ยท 2021-01-12
Assignee
Inventors
Cpc classification
G06T7/246
PHYSICS
G06V20/69
PHYSICS
G06T7/262
PHYSICS
International classification
G06T7/262
PHYSICS
Abstract
An information processing apparatus, an information processing method, and a cell analysis system are provided. The information processing apparatus includes a processor configured to: determine a frequency feature value based on motion data from an image of a cell, and control displaying information associated with the frequency feature value, wherein the frequency feature value includes a power spectral density for each time range and each frequency band, and wherein the information associated with the frequency feature value is displayed in association with the each time range and the each frequency band.
Claims
1. An analyzing system comprising: a processor configured to: calculate, for each time range, a feature value indicating a feature of an amount of movement in a target video image in which a target of analysis is imaged over time, wherein the feature is one of a mean value, a maximum value, a minimum value, a standard deviation, a variance, or a variation coefficient; and superimpose the feature value for each time range to a respective frame of the target video image, wherein the feature value is visualized by applying shading in accordance with a magnitude of the feature value such that the feature value is superimposed as the shading on the target video image to form a feature value-displaying video image.
2. The analyzing system according to claim 1, wherein the processor is configured to set, in a still image included in the target video image, a certain region, wherein the feature value is calculated for the certain region in the target video image.
3. The analyzing system according to claim 2, wherein the processor is configured to set a region that has a motion amount equal to or greater than a threshold as the certain region.
4. The analyzing system according to claim 1, wherein the target includes a cell.
5. The analyzing system according to claim 4, wherein the cell is a nerve cell.
6. The analyzing system according to claim 1, the shading applied for the feature value has a first color.
7. The analyzing system according to claim 6, wherein a second feature value indicating different second feature is visualized by applying shading in accordance with a second magnitude of the second feature value such that the feature value is superimposed as the shading on the target video image to for a second feature value-displaying image, and the shading applied for the second feature value has a second color.
8. A cell analysis system comprising: an imaging apparatus configured to image a cell over time; a processor configured to calculate, for each time range, a feature value indicating a feature of an amount of movement in a target video image obtained by the imaging apparatus, and superimpose the feature value for each time range to a respective frame of the target video image, wherein the feature value is visualized by applying shading in accordance with a magnitude of the feature value such that the feature value is superimposed as the shading on the target video image to form a feature value-displaying video image; and a display unit that displays a superimposed image, wherein the feature is one of a mean value, a maximum value, a minimum value, a standard deviation, a variance, or a variation coefficient.
9. The cell analysis system according to claim 8, wherein the imaging apparatus is a microscopy apparatus.
10. The cell analysis system according to claim 8, wherein the cell is a nerve cell.
11. The cell analysis system according to claim 8, wherein the processor is configured to set, in a still image included in the target video image, a certain region, wherein the feature value is calculated for the certain region in the target video image.
12. The cell analysis system according to claim 11, wherein the processor is configured to set a region that has a motion amount equal to or greater than a threshold as the certain region.
13. The cell analysis system according to claim 8, the shading applied for the feature value has a first color.
14. The cell analysis system according to claim 13, wherein a second feature value indicating different second feature is visualized by applying shading in accordance with a second magnitude of the second feature value such that the feature value is superimposed as the shading on the target video image to for a second feature value-displaying image, and the shading applied for the second feature value has a second color.
Description
BRIEF DESCRIPTION OF THE FIGURES
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
DETAILED DESCRIPTION
(26) Hereinafter, some embodiments of the present disclosure will be described with reference to the drawings.
First Embodiment
(27) An analyzing system according to a first embodiment of the present disclosure will be described.
(28)
(29) The video image acquisition unit 101 obtains a target video image which is a video image to analyze. A nonlimiting example of target video image is a video image of a cell or a group of cells as a target of analysis being imaged over time. The video image may include a video image made up of a plurality of frames continuously imaged, or still images from time-lapse imaging. The target video image may be obtained at a rate set appropriately depending on the target of analysis. In cases where the target of analysis is nerve cells, the rate may be 50 fps (frame/sec) or lower, and may be, for example, 1 fpm (frame/min). In cases where the target of analysis is cardiomyocytes, the rate may be 150 fps or more.
(30) The target video image may be a video image obtained by imaging using any of various optical imaging methods such as bright-field imaging, dark-field imaging, phase difference imaging, fluorescence imaging, confocal imaging, multiphoton excitation fluorescence imaging, absorbed light imaging and scattered light imaging.
(31)
(32) The motion detection unit 102 detects an amount of movement (motion vector) in the target video image (
(33) The range-specifying unit 103 specifies a calculation range in the target video image. The calculation range is a range for calculating a feature value, which will be described later, in the target video image. One or more calculation ranges may be provided.
(34) The range-specifying unit 103 may specify as the calculation range a range instructed by a user, or may specify a predetermined range as the calculation range. The predetermined range may be, for example, as shown in
(35) The feature value calculation unit 104 calculates a feature value for each calculation range (
(36) The feature value calculation unit 104 may calculate the feature value by using the time-motion waveform. Specific examples of the feature values include a pulsating area (area of the cells), and mean value, maximum value, minimum value, standard deviation, variance, and variation coefficient of amounts or directions of movement. The feature value calculation unit 104 continuously calculates the feature value while moving the time range to calculate the feature value in the time-motion waveform. The time range to calculate the feature value may be set appropriately depending on type of the feature value, motion of the target of analysis, or the like.
(37) Further, the feature value may include a frequency feature value. The feature value calculation unit 104 may calculate the frequency feature value by performing a frequency domain analysis of the time-motion waveform.
(38) The feature value calculation unit 104 performs fast Fourier transform (FFT) analysis of the waveform within the time-window while moving the time-window (
(39) The feature value calculation unit 104 may calculate mean intensity, peak frequency, mean power frequency (MPF) or the like, which can be obtained from the frequency domain analysis, as the frequency feature value (
(40) The feature value display unit 105 visualizes temporal change or spatial change of the feature value. As a specific example, the feature value display unit 105 may visualize the feature value and superimposes the visualized feature value on the target video image to form a feature value-displaying video image (
(41) The feature value-displaying video image shown in
(42) The feature value display unit 105 may generate the feature value-displaying video image by superimposing various feature values other than the MPF as well.
(43) In addition, the feature value display unit 105 may allow frequency characteristics for each time-window mentioned above to be displayed, to visualize spatial change of the feature value (
(44) GABA is a biologically active substance which functions as an inhibitory stimulus to nerve cells, and the nerve cells treated with GABA would show a motion with a low frequency (oscillation). Glutamic acid is a biologically active substance which functions as an excitatory stimulus to nerve cells, and the nerve cells treated with GABA would show a motion with a high frequency. In the frequency characteristics shown in
(45) Further, the feature value display unit 105 may display the feature value calculated by the feature value calculation unit 104 in a table or graph (
(46) The analyzing system 100 according to this embodiment is configured as described above. The analyzing system 100 enables the evaluation of motion of the target of analysis in the target video image, using the feature value. More specifically, the analyzing system 100 may be used in evaluation of effects of biologically active substances, effectiveness of drugs, evaluation of toxicity, quality control of nerve cells, evaluation of differentiation state of nerve cells, identification of abnormal cells and regions having abnormal networks, evaluation of pathological conditions by evaluating the cells derived from the pathological conditions, and the like.
(47) The target of analysis by the analyzing system 100 is not specifically limited. Examples of suitable targets of analysis by the analyzing system 100 include nerve cells. The movement (oscillation, etc.) of nerve cells is likely to be influenced by the kind of stimulus (inhibitory, excitatory, etc.) being applied to the nerve cells and by the state of the formation of neuronal networks. However, the movement of nerve cells is very small compared to pulsation of cardiomyocytes and the like, and it demands higher accuracy in the analysis. As the analyzing system 100 is able to evaluate the motion of the cells with high accuracy using the feature value as an index, nerve cells can be a suitable target of analysis using this analyzing system.
Second Embodiment
(48) An analyzing system according to a second embodiment of the present disclosure will be described.
(49)
(50) The video image acquisition unit 201 obtains a target video image which is a video image to analyze. A nonlimiting example of target video image is a video image of a cell or a group of cells as a target of analysis being imaged over time. The video image may include a video image made up of a plurality of frames continuously imaged, or still images from time-lapse imaging. The target video image may be obtained at a rate set appropriately depending on the target of analysis. In cases where the target of analysis is nerve cells, the rate may be 50 fps (frame/sec) or lower, and may be, for example, 1 fpm (frame/min). In cases where the target of analysis is cardiomyocytes, the rate may be 150 fps or more.
(51) The target video image may be a video image obtained by imaging using any of various optical imaging methods such as bright-field imaging, dark-field imaging, phase difference imaging, fluorescence imaging, confocal imaging, multiphoton excitation fluorescence imaging, absorbed light imaging and scattered light imaging (see
(52) The video image acquisition unit 201 may obtain the target video image from an imaging apparatus (microscopic imaging apparatus) (not shown in the drawing), or, it may obtain as the target video image a video image stored in storage or a video image provided from a network. At this time, the video image acquisition unit 201 may obtain the target video image by sampling, at a predetermined period depending on type of the target of analysis, from the video images which have been imaged in advance. The video image acquisition unit 201 provides the obtained target video image to the object region specifying unit 202.
(53) The object region specifying unit 202 specifies, in a still image included in the target video image (hereinafter referred to as a target still image), an analysis object region (
(54) The object region specifying unit 202 performs image processing on the target still image and specifies the analysis object region.
(55) The object region specifying unit 202 may specify the analysis object region by detection by dynamic ranges, matching or other image processing. At this time, the object region specifying unit 202 may select the target of analysis to detect as the analysis object region by the threshold. For example, it may select whether to detect cell bodies or neurites of nerve cells, or both of them.
(56) The motion detection unit 203 detects an amount of movement (motion vector) in the target video image (
(57) At this time, the motion detection unit 203 detects the amount of movement in the analysis object region (the set of analysis object sections D1) specified by the object region specifying unit 202. Specifically, the motion detection unit 203 detects the amount of movement in the target video image from one frame of target still image to the next frame of target still image, for each analysis object section D1 included in the analysis object region. The motion detection unit 203 is capable of converting the amount of movement of each analysis object section D1 into a motion velocity.
(58)
(59) As shown in
(60) The feature value calculation unit 204 calculates a feature value for the movement-containing region (
(61) The feature value calculation unit 204 may calculate a ratio of the movement-containing region as the feature value. The ratio of the movement-containing region means the ratio of the movement-containing region to the analysis object region, which may be, for example, the ratio of the motion-detected sections D2 to the analysis object sections D1. The ratio of the movement-containing region makes it possible to determine how much of the region that has been determined to have the target of analysis (analysis object region) is the region in which the movement has occurred (movement-containing region). This enables one to determine that, for example, in cases where the ratio of the movement-containing region is large, the whole target of analysis (cells, etc.) is oscillating; and in cases where the ratio of the movement-containing region is small, a specific part of the target of analysis is oscillating.
(62) In addition, the feature value calculation unit 204 may calculate an analysis object region velocity as the feature value. The analysis object region velocity is a mean value of motion velocities of the analysis object region, which can be calculated by averaging the motion velocities of the respective sections D1 (including the motion-detected sections D2). The analysis object region velocity is a mean value of the motion velocities of the whole target of analysis, which enables one to determine overall motion velocity of the target of analysis. By averaging the motion velocities of the region limited to the analysis object region, it can avoid averaging the motion velocities of the regions where the target of analysis does not exist (intervals between the cells, etc.).
(63) The feature value calculation unit 204 may calculate a movement-containing region velocity as the feature value. The movement-containing region velocity is a mean value of motion velocities of the movement-containing region, which can be calculated by averaging the motion velocities of the respective motion-detected sections D2. The movement-containing region velocity is a mean value of the motion velocity of the moving parts of the target of analysis. For example, in cases where only a specific part in the target of analysis is vigorously oscillating, the movement-containing region velocity enables one to determine the motion velocity of this specific part alone. If a mean value of the motion velocities of the whole target of analysis was to be calculated, the motion velocities of the parts without movement would be averaged with it. In view of this, the movement-containing region velocity may be useful especially in cases where only a specific part of the target of analysis is moving.
(64) Further, the feature value calculation unit 204 may calculate a frequency feature value as the feature value. The feature value calculation unit 204 may calculate the frequency feature value by calculating a time-motion waveform (see
(65) The feature value calculation unit 204 performs fast Fourier transform (FFT) analysis of the waveform within the time-window while moving the time-window (
(66) As shown in the figure, the feature value calculation unit 204 may calculate an area of a predetermined frequency band of the power spectral density (PSD area) as the frequency feature value. The frequency band to calculate the PSD area may be set appropriately depending on the frequency of the oscillation to observe, and may be, for example, from 0 to 0.1 Hz or less.
(67)
(68) Thus, with the feature value calculation unit 204 calculating the PSD area of the predetermined frequency band, it is possible to extract, from the oscillation of the target of analysis, the oscillation at only a frequency band of interest. It is able to omit from the analysis the oscillations of the different frequencies which are unrelated.
(69) In the same manner as in the first embodiment, the feature value calculation unit 204 may calculate mean intensity, peak frequency, mean power frequency (MPF) or the like, which can be obtained from the frequency domain analysis, as the frequency feature value (
(70) The feature value display unit 205 visualizes temporal change or spatial change of the feature value. As a specific example, the feature value display unit 205 may visualize the feature value and superimposes the visualized feature value on the target video image to form a feature value-displaying video image (
(71)
(72) Further, the feature value display unit 205 may display the feature value calculated by the feature value calculation unit 204 in a table or graph (
(73) The analyzing system 200 according to this embodiment is configured as described above. The analyzing system 200 enables the evaluation of motion of the target of analysis in the target video image, using the feature value. More specifically, the analyzing system 200 may be used in evaluation of effects of biologically active substances, effectiveness of drugs, evaluation of toxicity, quality control of nerve cells, evaluation of differentiation state of nerve cells, identification of abnormal cells and regions having abnormal networks, evaluation of pathological conditions by evaluating the cells derived from the pathological conditions, and the like.
(74) The target of analysis by the analyzing system 200 is not specifically limited. Examples of suitable targets of analysis by the analyzing system 200 include nerve cells. The movement (oscillation, etc.) of nerve cells is likely to be influenced by the kind of stimulus (inhibitory, excitatory, etc.) being applied to the nerve cells and by the state of the formation of neuronal networks. However, the movement of nerve cells is very small compared to pulsation of cardiomyocytes and the like, and it demands higher accuracy in the analysis. As the analyzing system 200 is able to evaluate the motion of the cells with high accuracy using the feature value as an index, nerve cells can be a suitable target of analysis using this analyzing system.
(75) In addition, this embodiment extracts as the analysis object region the region where the target of analysis exists in the target video image and performs the analysis of the amount of movement and the calculation of the feature value for the analysis object region. In nerve cells, their parts which may oscillate are relatively local compared to those of other cells such as cardiomyocytes, and they can be effectively analyzed using the analysis object region of the analyzing system 200 according to this embodiment. Further, since the analyzing system 200 uses the area of the specific frequency band of the power spectral density in the analysis, it is able to extract from neurites and cell bodies of nerve cells, or the like, respective oscillations of different frequencies. In this respect, it is suitable for the analysis of nerve cells.
(76) The present disclosure is not limited to each of the foregoing embodiments but can be modified within the scope without departing from the gist of the present disclosure.
(77) The present disclosure may employ the following configurations.
(78) (1) An analyzing system including:
(79) a feature value calculation unit configured to calculate, for each time range, a feature value indicating a feature of an amount of movement in a target video image in which a target of analysis is imaged over time.
(80) (2) The analyzing system according to (1), further including:
(81) a feature value display unit configured to visualize temporal change or spatial change of the feature value.
(82) (3) The analyzing system according to (2), in which
(83) the feature value display unit is configured to visualize temporal change of the feature value and superimpose the visualized temporal change of the feature value on the target video image to form a feature value-displaying video image.
(84) (4) The analyzing system according to any one of (1) to (3), further including:
(85) a range-specifying unit configured to specify a certain range of the target video image as a calculation range;
(86) the feature value calculation unit being configured to calculate the feature value for each calculation range.
(87) (5) The analyzing system according to any one of (1) to (4), in which
(88) the feature value is any one selected from the group consisting of mean value, maximum value, minimum value, standard deviation, variance, and variation coefficient of amounts or directions of movement; a frequency feature value; or a combination thereof.
(89) (6) The analyzing system according to any one of (1) to (5), in which
(90) the frequency feature value is mean intensity, peak frequency or mean power frequency, obtained from a frequency domain analysis.
(91) (7) The analyzing system according to any one of (1) to (6), which
(92) analyzes a nerve cell as the target of analysis.
(93) (8) The analyzing system according to any one of (1) to (7), further including:
(94) an object region specifying unit configured to specify, in a still image included in the target video image, an analysis object region which is a region where the target of analysis exists;
(95) the feature value calculation unit being configured to calculate the feature value for the analysis object region in the target video image.
(96) (9) The analyzing system according to (8), in which
(97) the feature value calculation unit is configured to calculate the feature value in the analysis object region using a movement-containing region which is a region that has a motion velocity equal to or greater than a threshold.
(98) (10) The analyzing system according to (9), in which
(99) the feature value calculation unit is configured to calculate, as the feature value, a ratio of the movement-containing region to the analysis object region.
(100) (11) The analyzing system according to (9) or (10), in which
(101) the feature value calculation unit is configured to calculate, as the feature value, a mean value of motion velocities of the analysis object region.
(102) (12) The analyzing system according to any one of (9) to (11), in which
(103) the feature value calculation unit is configured to calculate, as the feature value, a mean value of motion velocities of the movement-containing region.
(104) (13) The analyzing system according to any one of (1) to (12), in which
(105) the feature value calculation unit is configured to calculate, as a frequency feature value, an area of a predetermined frequency band of a power spectral density obtained from a frequency domain analysis of the amount of movement.
(106) (14) An analyzing program which causes a computer to function as:
(107) a feature value calculation unit configured to calculate, for each time range, a feature value indicating a feature of an amount of movement in a target video image in which a target of analysis is imaged over time.
(108) (15) An analyzing method including:
(109) calculating, for each time range, a feature value indicating a feature of an amount of movement in a target video image in which a target of analysis is imaged over time.
(110) It should be understood that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the present subject matter and without diminishing its intended advantages. It is therefore intended that such changes and modifications be covered by the appended claims.