Feedback feeding system for pond recirculation with fusion of machine vision and infrared detection

Abstract

Disclosed is a feedback feeding system pond recirculation with fusion of machine vision and infrared detection. Feeding strategies are provided according to the feeding behavior of fish in cooperative with machine vision technology and infrared sensing technology. The pond recirculating aquaculture is an open environment. The change of natural environment may affect the feedback signal of the fish feeding behavior provided by the machine vision due to the low visibility; or it may affect the feedback signal of the fish feeding behavior provided by the infrared detection because low temperature may decrease feeding intensity. The fusion of these two strategies can solve the problem of intelligent feeding strategies in the pond recirculating aquaculture mode.

Claims

1. A feeding system, comprising a rack configured to mount at an inlet of a pond on a pond shore, a controller and a feeder mounted on the rack and configured to be located over the pond; wherein the controller comprises a control box, a development board, a night vision fisheye wide-angle camera, a plurality of night vision camera infrared fill lights, a plurality of infrared sensors, and a mechanical infrared sensor assister; the development board is arranged within the control box, and is connected to the night vision fisheye wide-angle camera and the infrared sensor; the night vision fisheye wide-angle camera is mounted on a bottom of the control box, and is connected to the night vision camera infrared fill light to provide an image stream of the fish feeding behavior; the development board is configured to calculate intensity E.sub.K of fish feeding behavior based on a reflective area of a water surface according to change characteristics of the reflective area reflected by the image stream; two night vision camera infrared fill lights are arranged on both sides of the night vision fisheye wide-angle camera, respectively; four infrared sensors are mounted within the control box, and each two adjacent infrared sensors form an angle of 90°; the infrared sensors together with the mechanical infrared sensing assister provide monopulse signals to the development board; the development board is configured to calculate intensity E.sub.J of fish feeding behavior based on intensity of the water fluctuation according to intensity of water fluctuation of the pond reflected by the monopulse signals; the feeder comprises a feed storage pin, a screw conveyor, a down pipe, a feed distributing pan and a motor; the feed in the feed storage pin is capable of being conveyed to the down pipe under the drive of the motor through the screw conveyor; the feed distributing pan is capable of distributing the feed to the pond recirculating aquaculture pond evenly; the motor is connected to the development board; the development board is configured to calculate intensity E of the fish feeding behavior according to the obtained E.sub.K and E.sub.J:
E=(1−ε)E.sub.K+εE.sub.J wherein, ε represents the coordination coefficient of feeding strategies; when E is greater than the present feeding strategy threshold G, the development board controls the rotation of the motor to starting feeding.

2. The feeding system according to claim 1, wherein the mechanical infrared sensing assistor comprises a mechanical infrared sensing auxiliary joint, an infrared sensing feedback adjustment knob, and a mechanical infrared sensing auxiliary rod; the mechanical infrared sensing auxiliary joint comprises an upper sub-joint and a lower sub-joint; the lower sub-joint is provided with a hole thereon; the mechanical infrared sensing auxiliary rod is inserted into the hole from top to bottom; the upper end is located inside the mechanical infrared sensing auxiliary joint; the infrared sensing feedback adjustment knob is installed on the upper sub-joint; the end of the feedback adjustment knob is abutted against the upper end of the mechanical infrared sensing auxiliary rod; the swing amplitude of the mechanical infrared sensing auxiliary rod caused by the fish feeding behavior is controlled by the tensioning effect to adjust the infrared sensing feedback.

3. The feeding system according to claim 1, wherein E.sub.K is calculated by the following steps: a. transmitting, by the night vision fisheye wide-angle camera, the real-time image stream of the fish feeding behavior to the development board, wherein the image stream is RGB space; b. converting, by the development board, the image stream to HSV space; c. extracting the reflective area of the water surface of the pond recirculating aquaculture pond in the image stream from the following expression: $f\left(x,y\right)=\{\begin{array}{cc}1& \left(I_s\left(x,y\right)<T_s\mathrm{and}{I}_v\left(x,y\right)>T_v\right)\\ 0& \mathrm{others}\end{array}$ wherein, I.sub.s(x, y) and I.sub.v(x, y) represents the saturation and brightness of the image at (x, y), respectively; T.sub.s and T.sub.s represents the saturation threshold and brightness threshold, respectively; f(x, y) represents the value of pixel (x, y) after binarization processing; d. extracting the characteristics information of the fish feeding behavior in the image stream by Lucas-Kanade optical flow algorithm; the velocity change amplitude of the motion vector in two adjacent image streams are calculated according to the following expression: $v=\frac{{\Sigma }_{x,y}.Math.f\left(x,y\right).Math.}{N}$ wherein, (x, y) represents the coordinator of the reflective area of the current frame; N represents the total number of motion vectors in the current frame; e. calculating the velocity change amplitude distribution of the motion vector in two adjacent image streams are calculated; the distribution probability of different velocity intervals according to the following expression:
P(i)=(H(i)/N)(0≤i≤m) wherein, m represents the number of intervals v being divided into; H(i) and P(i) represents the number and probability of motion vectors falling into a certain speed interval; f. measuring, based on the information entropy, the degree of irregularity in the distribution probability of the reflective area change on the water surface of the pond recirculating aquaculture pond; the expression is as follows:
C.sub.E=Σ.sub.j=1.sup.mP(j)log.sub.2(P(j)) g. calculating E.sub.K by the following expression: $E_K=L\times C_E\times v^2=-L\underset{j=1}{\overset{m}{.Math.}}P\left(j\right)\times {\left(\frac{{\Sigma }_{x,y}.Math.f\left(x,y\right).Math.}{N}\right)}^2$ wherein, L is the light correction factor.

4. The feeding system according to claim 1, E.sub.J is calculated by the following steps: a. detecting irregulate rotation of the mechanical infrared sensing auxiliary joint caused by water fluctuation of the pond recirculating aquaculture pond caused by feeding activity of shoals of fish; b. converting, by the infrared sensors installed inside the control box, the irregular rotation into a monopulse signal, marked with the following expressions: $S\left(i\right)=\{\begin{array}{cc}\ln s_j,& {\Sigma }_0^t{s}_j>M\\ 0& \mathrm{others}\end{array}i=1,2,3,4;j=1,2,3\mathrm{.Math.}\mathrm{.Math.},t$ wherein, S(i) represents the sensing state of the i-th sensor of the four infrared sensors; t represents the time period in seconds; s.sub.j represents the number of monopulse of a single infrared sensor per second in period t; M presents sensing threshold of the infrared sensor; c. calculating E.sub.J by the following expressions: $E_J=F\underset{i=1}{\overset{4}{.Math.}}S\left(i\right)$ wherein, F represents the water fluctuation correction coefficient.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a schematic diagram of a feeding system according to one embodiment of the present disclosure.

(2) FIG. 2 is a schematic diagram of a controller of the feeding system according to one embodiment of the present disclosure.

(3) FIG. 3 is a cross-sectional diagram of a mechanical infrared sensing assistor of the feeding system according to one embodiment of the present disclosure.

(4) FIG. 4 is a schematic diagram of a feeder of the feeding system according to one embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE DISCLOSURE

(5) The present disclosure will be described in detail below in conjunction with the drawings. The following specific embodiments are used to illustrate the present invention, but not to limit the scope of the present invention.

(6) The present disclosure provides a feedback feeding system pond recirculation with fusion of machine vision and infrared detection aiming to solve the problem in the prior art. The system is cost-effective and accurate in feeding, realizing the intelligent feeding strategies in pond recirculating aquaculture.

(7) The present disclosure is described as follows.

(8) The present disclosure determines the intensity E of the fish feeding behavior in two aspects. On the one hand, the intensity of the fish feeding behavior is determined by the change characteristics of the reflective area of the water surface of the pond recirculating aquaculture pond. In the present disclosure, E.sub.K represents the fish feeding behavior based on the reflective area of the water surface. On the other hand, the intensity of the fish feeding behavior is determined by the water fluctuation of the pond recirculating aquaculture pond. In the present disclosure, E.sub.J represents the fish feed behavior based on the intensity of the water fluctuation. The fusion of these two aspects will affect the final intelligent feeding strategy. When the reflective area of the water surface caused by the natural environment is not clear, the generation of feeding strategy relies more on the infrared sensing feedback. When the water fluctuation caused by low temperature environment, the generation of feeding strategy relies more on the machine vision.

(9) More detail embodiments will be described with the following examples.

(10) As shown in FIG. 1, the feeding system with fusion of machine vision and infrared detection comprises a rack 1, a controller 2 and a feeder 3.

(11) The rack 1 is mounted on pond shore of the inlet of the pond recirculating aquaculture pond. The rack 1 provides a stable work environment for the controller 2 and the feeder 3 which are mounted thereon.

(12) As shown in FIG. 2, the controller 2 comprises a control box 4, a control box cover 5, a power supply 6, a development board 7 which is Raspberry Pi® 3B+, a 7-inch HDMI capacitive touch screen 8, a night vision fisheye wide-angle camera 9, a night vision camera infrared fill light 10, an infrared sensor 11, and a mechanical infrared sensor assister 12. The top of the control box 4 is receivable for the control box cover 5. The control box 4 can provide a dry and clean work environment for the power supply 6, the development 7, the touch screen 8, the night vision fisheye wide-angle camera 9, the night vision camera infrared fill light 10, the infrared sensor 11 and the mechanical infrared sensing assistor 12. The power supply 6 is arranged within the control box 4 to provide electric power for the development 7, the touch screen 8, the night vision fisheye wide-angle camera 9, the night vision camera infrared fill light 10 and the infrared sensor 11. The development board 7 is arranged within the control box 4 and is connected to the touch screen 8, the night vision fisheye wide-angle camera 9 and the infrared sensor 11. The fusion of the machine vision and the infrared detection is the core electrical device of the feeding system. The touch screen 8 is installed on the on the main viewing surface of the control box 4, and displays the interface of the feeding system. The touch screen 8 allows operation on the system, and the interface thereof may be programmed by web. Varies of smart devices (e.g. mobile phone, pad and computer, etc.) in local network can be remotely controlled by entering the Raspberry Pi IP address, which is convenient for users to operate and read. The night vision fisheye wide-angle camera 9 is installed at the bottom of the control box 4 and connected to the night vision camera infrared fill light 10 to provide the development board 7 with image streams of fish feeding behavior. The development board 7 calculates E.sub.K according to the change characteristics of the reflective area of the pond recirculating aquaculture pond reflected by the image stream, and then cooperates with the infrared sensor feedback feeding strategy to control the working state of the screw conveyor 18 in the feeder 3 to achieve the intelligent feeding. The night vision camera infrared fill lights 10 are connected on both sides of the night vision fisheye wide-angle camera 9 to provide a stable background for the image stream of the collected fry.

(13) The night vision fisheye wide-angle camera is mounted on the bottom of the control box, and is connected to the night vision camera infrared fill light to provide image stream of the fish feeding behavior. The development board is configured to calculates the intensity E.sub.K of fish feeding behavior based on the reflective area of the water surface according to the change characteristics of the reflective area of the pond recirculating aquaculture pond reflected by the image stream. Two night vision camera infrared fill lights are arranged on both sides of the night vision fisheye wide-angle camera, respectively. Four infrared sensors are mounted within the control box, and each two adjacent infrared sensors form an angle of 90°. The infrared sensors are cooperative with the mechanical infrared sensing assister to provide monopulse signal to the development board. The development board is configured to calculate the intensity E.sub.J of fish feeding behavior based on the intensity of the water fluctuation according to the intensity of the water fluctuation of the pond recirculating aquaculture pond reflected by the monopulse signal.

(14) When the intensity of the fish feeding behavior is determined according to the change characteristics of the reflective area of the water surface of the pond recirculating aquaculture pond, the calculation of E.sub.K is detailed as follows.

(15) a. The night vision fisheye wide-angle camera 9 transmits the real-time image stream of the fish feeding behavior to the development board 7, wherein the image stream is RGB space.

(16) b. The development board 7 converts the image stream to HSV space.

(17) c. The reflective area of the water surface of the pond recirculating aquaculture pond in the image stream is extracted from the following expression:

(18) $f\left(x,y\right)=\{\begin{array}{cc}1& \left(I_s\left(x,y\right)<T_s\mathrm{and}{I}_v\left(x,y\right)>T_v\right)\\ 0& \mathrm{others}\end{array}$

(19) wherein, I.sub.s(x, y) and I.sub.v(x, y) represents the saturation and brightness of the image at (x, y), respectively. T.sub.s and T.sub.s represents the saturation threshold and brightness threshold, respectively. f(x, y) represents the value of pixel (x, y) after binarization processing.

(20) d. The characteristics information of the fish feeding behavior in the image stream is extracted by Lucas-Kanade optical flow algorithm. The velocity change amplitude of the motion vector in two adjacent image streams are calculated according to the following expression:

(21) $v=\frac{{\Sigma }_{x,y}.Math.f\left(x,y\right).Math.}{N}$

(22) wherein, (x, y) represents the coordinator of the reflective area of the current frame; N represents the total number of motion vectors in the current frame.

(23) e. The velocity change amplitude distribution of the motion vector in two adjacent image streams are calculated. The distribution probability of different velocity intervals according to the following expression:
P(i)=(H(i)/N)(0≤i≤m)

(24) wherein, m represents the number of intervals v being divided into; H(i) and P(i) represents the number and probability of motion vectors falling into a certain speed interval.

(25) f. The information entropy is used to measure the degree of irregularity in the distribution probability of the reflective area change on the water surface of the pond recirculating aquaculture pond, so as to realize the analysis of the degree of irregularity in the fish feeding behavior. The expression is as follows:
C.sub.E=Σ.sub.j=1.sup.mP(j)log.sub.2(P(j))

(26) g. E.sub.K is calculated by the following expression:

(27) $E_K=L\times C_E\times v^2=-L\underset{j=1}{\overset{m}{.Math.}}P\left(j\right)\times {\left(\frac{{\Sigma }_{x,y}.Math.f\left(x,y\right).Math.}{N}\right)}^2$

(28) wherein, L is the light correction factor.

(29) In some embodiments, the calculation of E.sub.J is detailed as follows.

(30) a. The water fluctuation of the pond recirculating aquaculture pond caused by the feeding activity of the shoals of fish causes the mechanical infrared sensing auxiliary rod 15 to rotate irregularly in the mechanical infrared sensing auxiliary joint 13.

(31) b. The infrared sensors 11 installed inside the control box 4 converts the irregular rotation into a monopulse signal, marked with the following expressions:

(32) $S\left(i\right)=\{\begin{array}{cc}\ln s_j,& {\Sigma }_0^t{s}_j>M\\ 0& \mathrm{others}\end{array}i=1,2,3,4;j=1,2,3\mathrm{.Math.}\mathrm{.Math.},t$

(33) wherein, S(i) represents the sensing state of the i-th sensor of the four infrared sensors; t represents the time period in seconds; s.sub.j represents the number of monopulse of a single infrared sensor per second in period t; M presents sensing threshold of the infrared sensor 11. In the present disclosure, four infrared sensors 11 are installed on the infrared sensor rack within the control box 4, and two adjacent infrared sensors 11 form a 90° angle.

(34) c. E.sub.j is calculated by the following expressions:

(35) 0$E_J=F\underset{i=1}{\overset{4}{.Math.}}S\left(i\right)$

(36) wherein, F represents the water fluctuation correction coefficient.

(37) Feeding strategies are provided according to the feeding behavior of fish in cooperative with machine vision technology and infrared sensing technology. The calculation of E plays an important role. According to the obtained E.sub.K and E.sub.J, the intensity E of fish feeding behavior is calculated as follows:
E=(1−ε)E.sub.K+εE.sub.J

(38) wherein, ε represents the coordination coefficient of feeding strategies. When E is greater than the present feeding strategy threshold G, the development board controls the rotation of the motor to starting feeding.

(39) The foregoing are only specific embodiments of the present disclosure, and various changes and modifications made without departing from the concept and scope of the present disclosure, and all equivalent technical solutions also belong to the scope of the present invention.