Sky monitoring system

Abstract

In a sky monitoring system, comprising an image sensor, a wide-angle lens, a microprocessor, and a memory unit, wherein the sky monitoring system is configured to take pictures of a sky scene, wherein the sky monitoring system is configured to subdivide each picture of the sky scene into a group of patches and to determine one luminance value for each patch, wherein the sky monitoring system is configured to calculate an output based on the luminance values of the patches, the image sensor, the wide-angle lens, the microprocessor and the memory unit are integrated into one single sky monitoring device thus making the sky monitoring system an embedded system.

Claims

1. A sky monitoring system, comprising: an image sensor, a wide-angle lens positioned relative to the image sensor to direct light thereto, a microprocessor communicatively coupled to the image sensor, and a memory unit communicatively coupled to the microprocessor, wherein the image sensor, the wide-angle lens, the microprocessor, and the memory unit are integrated into one single sky monitoring device, and wherein a spectral response of the image sensor is corrected according to a CIE photopic luminosity function V(λ), wherein the microprocessor, using the memory unit, is configured to: capture a plurality of pictures of a sky scene; subdivide each of the plurality of pictures of the sky scene into a group of patches, wherein each patch corresponds to a unique incident angle of observed light rays from the sky; determine a luminance value for each of the patches; calculate an output based on the luminance values of the patches; and supply as the output an illuminance map of a surface inside a building.

2. The sky monitoring system of claim 1, wherein the image sensor is a CMOS image sensor, wherein the CMOS image sensor comprises between 3 million and 7 million pixels, wherein the CMOS image sensor features a signal-to-noise ratio between 20 dB and 60 dB, wherein the CMOS image sensor preferably features a 12-bits analog-to-digital converter, wherein the CMOS image sensor is preferably adapted to operate between −40° C. and 80° C.

3. The sky monitoring system of claim 1, wherein the wide-angle lens has a focal length between 1 mm and 4 mm.

4. The sky monitoring system of claim 1, wherein a short-pass filter and a long-pass filter are arranged between the wide-angle lens and the image sensor.

5. The sky monitoring system of claim 1, further comprising an opaque shield positioned between the wide-angle lens and a sun; and a motor, wherein the opaque shield is movable between an open position and a closed position, wherein the microprocessor, using the memory unit, further is configured to: move the opaque shield to the open position at a moment of image capturing; and then, move the opaque shield to the closed position in order to shield the lens and the image sensor from the sun.

6. The sky monitoring system of claim 1, wherein the sky monitoring device has maximum dimensions of 20 cm×15 cm×10 cm.

7. The sky monitoring system of claim 1, wherein the microprocessor comprises a field-programmable gate array chip.

8. The sky monitoring system of claim 1, wherein the microprocessor, using the memory unit, further is configured to do at least one of the following: supply as the output a current sky condition; supply as the output a current solar intensity; or supply as the output a current solar position.

9. The sky monitoring system of claim 1, wherein the microprocessor is configured generate the illuminance map of the surface inside the building based on a backward ray-tracing algorithm.

10. The sky monitoring system of claim 1, wherein the microprocessor outputs the illuminance map of the surface inside the building based on (i) a monitored luminance of the sky and ground half hemisphere, and (ii) the scene of the office room.

11. The sky monitoring system of claim 1, wherein the calculation comprises ray-tracing analysis.

12. A method of using a sky monitoring system, comprising: capturing, via an image sensor of the sky monitoring system, a plurality of pictures of a sky scene using a sky monitoring device; correcting a spectral response of the image sensor according to a CIE photopic luminosity function V(λ); subdividing, using a microprocessor, each of the plurality of pictures of the sky scene into a group of patches, wherein each patch corresponds to a unique incident angle of observed light rays from the sky; determining a luminance value for each of the patches; calculating an output based on the luminance values of the patches; and supplying as the output an illuminance map of a surface inside the building.

13. The method of claim 12, further comprising at least one of: performing photo-realistic rendering using the output; controlling lighting using the output; solar position tracking using the output; or monitoring solar intensity using the output.

14. The method of claim 12, wherein the image sensor is a CMOS image sensor, wherein the CMOS image sensor comprises between 3 million and 7 million pixels, wherein the CMOS image sensor features a signal-to-noise ratio between 20 dB and 60 dB, wherein the CMOS image sensor preferably features a 12-bits analog-to-digital converter, wherein the CMOS image sensor is preferably adapted to operate between −40° C. and 80° C.

15. The method of claim 12, wherein a short-pass filter and a long-pass filter are arranged between the wide-angle lens and the image sensor.

16. The method of claim 12, wherein the sky monitoring system has maximum dimensions of 20 cm×15 cm×10 cm.

17. The method of claim 12, wherein sky monitoring system comprises a microprocessor and a memory unit.

18. The method of claim 12, wherein the illuminance map of the surface inside the building is based on a backward ray-tracing algorithm.

19. The method of claim 12, wherein the illuminance map of the surface inside the building is based on (i) a monitored luminance of the sky and ground half hemisphere, and (ii) the scene of the office room.

20. The method of claim 12, wherein the calculation comprises ray-tracing analysis.

Description

FIGURES

(1) In the following, example embodiments of the disclosure are described in detail by means of drawings, wherein the drawings show:

(2) FIG. 1: a schematic block diagram of a sky monitoring system;

(3) FIG. 2: a schematic block diagram of an FPGA microprocessor;

(4) FIG. 3: a sky monitoring system installed outside the window of a building and facing the sky; and

(5) FIG. 4: a photo-realistic rendering of a reference office room created by a sky monitoring system.

DETAILED DESCRIPTION

(6) In a preferred embodiment, the sky monitoring system is an embedded system integrating quasi real-time lighting computation with sky monitoring and luminance mapping, which can also be used in photo-realistic rendering, evaluation of lighting performance parameter and building automation. The embedded system comprises an SoC (system-on-chip) FPGA (field-programmable gate array) chip and a calibrated low-cost image sensor with a wide-angle lens. The embedded system is configured to employ the high dynamic range (HDR) technique to make the luminance detection range of the embedded system cover 7 orders of magnitude (121 cd/m.sup.2 to 3×10.sup.9 cd/m.sup.2) including the sun, sky and environmental objects during daytime. The monitored luminance is mapped to 1.23 million sky patches and then the FPGA chip conducts the lighting computation based on ray-tracing, for example by means of the RADIANCE program. This lighting computation is carried out on board; in other words: the sky monitoring system is configured to conduct lighting computation by means of ray-tracing on board and/or without the use of external computation devices.

(7) FIG. 1 shows a schematic block diagram of a sky monitoring system S according to the disclosure. The sky monitoring system S comprises an image sensor 1, a wide-angle lens 16 which is optically connected to the image sensor 1, a field programmable gate array (FPGA) chip 3, an internal memory 11, an SD memory 18 and several outputs 19. All these components are integrated on a printed circuit board (PCB) 17. All these components thus form a sky monitoring device which constitutes the sky monitoring system S. In other words: in this embodiment, the sky monitoring system S is an integrated sky monitoring device having all its components embedded on the PCB 17. The internal memory 11 and/or the SD memory 18 is preferably a 1 GB memory. FIG. 1 thus shows a highly integrated embedded sky monitoring system S independent of any external computation platform.

(8) In the embodiment of the disclosure shown in FIG. 1, a low-cost digital image sensor 1 is employed as a major detector in this platform. The image sensor 1 comprises a 1/2.5 inch complementary metal oxide semiconductor (CMOS) image sensor featuring 5 million pixels (2592×1944), 38.1 dB signal-to-noise ratio (SNR) and 12-bits analog to digital converter (ADC). This image sensor 1 is a low power consuming device with maximum power at 381 milli-Watts (mW). A wide operating temperature from −30° C. to 70° C. allows it to work in most geographical locations and seasons on earth.

(9) The image sensor 1 is coupled with the wide-angle lens 16 of focal length at 2.5 millimeter (mm), ensuring that the field of view (FOV) of the sky monitoring system S covers a major area of a monitored sky dome and surrounding landscape. The reason why a wider-angle lens may not be employed in this particular embodiment lies in the consideration of keeping the distortion at a tolerable scale, as extreme wide-angle lenses tend to suffer from apparent radical and tangential distortions, which aggravates complexity in post-calibration. For the sky monitoring device in this embodiment, the imaging system spans a view angle of 129.8°×96.8° in horizontal and vertical directions, respectively, and the maximum is 162.5° in the diagonal direction.

(10) In a preferred embodiment, a short-pass filter and a long-pass filter are placed between the wide-angle lens 16 and the image sensor 1 to correct its spectral response according to the CIE photopic luminosity function V(λ), thus making the image sensor 1 photometrically correct (f1′ error 8.5%). The filters are not shown in FIG. 1.

(11) In a preferred embodiment, the vignette and geometric distortions of the wide-angle lens 16 are measured and calibrated to ensure mapping accuracy.

(12) In a further preferred embodiment, considering the extreme intensity of the solar rays, an opaque shield is positioned in front of the wide-angle lens 16 to protect the image sensor 1 from incoming solar rays (not shown in FIG. 1). Preferably, the opaque shield is controlled by a stepper motor to automatically open the opaque shield at the moment of image capturing and close the opaque shield again when the image capturing is finished. The maximum open time is preferably 0.5 seconds, thus securing that the image sensor 1 will not be overheated during this short period by the intensive direct solar rays, thus limiting the risk of irreversible physical damage to pixels of the image sensor 1.

(13) In the embodiment shown in FIG. 1, the FPGA chip 3 and the 1-Gigabyte external SD memory 18 constitute a hub for logic control and computation. As an embedded system, its power dissipation, compactness and computing power are among the major concerns. Due to its highly customizable feature as an economical solution in non-recurring engineering, a single FPGA-SoC (system on chip) microprocessor is chosen to accomplish tasks including precise control of sensors and actuators, synthesis in HDR imaging, luminance mapping and on-board lighting computation. The FPGA chip 3 is for example an Altera Cyclone V chip.

(14) FIG. 2 shows in more detail the FPGA chip 3 already shown in FIG. 1. The FPGA chip 3 integrates two major parts on a single silicon die: a FPGA fabric part 4 and a hard core processor (HPS) part 5. The FPGA fabric part 4 is a massive area of programmable gate arrays (85K logic cells) preferable in parallel computing and control of sensors. In this embodiment, the FPGA fabric part 4 is limited to the control of image sensor 1 and stepper motor 2 for occluding the wide-angle lens 16 from the sky or the sun. The HPS part 5 is mainly composed of an ARM processing unit 9 (in particular an ARM Cortex-A9 dual core processor working at 925 MHz) together with communication bridges 14 to the FPGA fabric part 4. In this embodiment, the HPS part 5 is designed to be in charge of relatively high-level algorithms including the synthesis in HDR imaging, luminance mapping of the sky scene and lighting simulation.

(15) FIG. 2 also shows that the FPGA fabric part 4 comprises a peripheral controller 6 with a hardware interface 7 and a computation accelerator 8. The peripheral controller 6 is connected to the image sensor 1 and the motor and actuator 2 of the opaque shield (not shown). The peripheral controller 6 is linked via a bridge 15 of the FPGA fabric part 4 to an SDRAM controller 10 of the HPS part 5. The SDRAM controller 10 is linked to the Memory (RAM) 11.

(16) The FPGA fabric part 4 furthermore comprises an input/output interface 12, connected to the HPS part 5 via the communication bridges 14. The input/output interface 12 is connected to a multitude of buttons 13.

(17) The FPGA fabric part 4 feeds the image sensor 1 with a high frequency clock operating at 96 MHz to drive a sequential output of pixels. Together with multiple internal counters, the processor is able to control the exposure time of the image sensor 1 ranging from 10 nanoseconds up to minutes. Based on the consideration of capturing images of rapid moving clouds, the maximum exposure time is typically limited to 400 milli-seconds (ms) to avoid blurring. As multiple clocks are working inside the FPGA chip 3, the minimum exposure time is typically set to be 1 micro-second (s) to reduce the effect of clock domain crossing (CSC), thus ensuring accuracy of minimum integration time.

(18) Although the 12-bits ADC of the image sensor outputs 1 to 4096 levels of discrete gray-scale values, one embodiment of the disclosure filters out values outside the linear region of pixels and keeps only 50 to 3400 to be the valid range, at the same time, excluding dark current noise and reset noise of the sensor. In this way, the brightest pixel receives 64 times more photons than that of the dimmest pixel on the same frame. In a preferred embodiment, five consecutive frames are taken with exposure times of 1 μs, 10 μs, 655 μs, 42 ms and 400 ms, respectively. The synthesized luminance spans 7 orders of magnitude (150 dB). This is sufficient to cover the range of luminance of the direct solar rays (3×10.sup.9 cd/m.sup.2) and that of environmental objects (121 cd/m.sup.2) in daytime.

(19) In a preferred embodiment, every four pixels of RGB channels are merged as a group, and the device is thus able to output 1.23 million cells of luminance values, wherein each cell corresponds to a unique directional angle of an incident ray. Therefore, it is possible to map the luminance values from the image plane to 1.23 million patches of the sky or ground dome. As the focal length of the wide-angle lens 16 is infinitesimal compared to the distance between the sky monitoring device and environmental objects (such as neighboring buildings, gardens and static vehicles), it is possible to approximate both ambient objects and the sky to be at infinite distance from the wide-angle lens 16 and virtually immerse the environmental objects into the infinite sky and ground dome, regarding them, including clouds and the sun, to be one integrated entity in mapping.

(20) The microprocessor of the sky monitoring system in the embodiment shown in FIGS. 1 and 2, namely the FPGA chip 3, is able to accomplish the control of HDR imaging and luminance mapping within one second. The mapped luminance of the sky and objects is stored in a calculation file as input to the RADIANCE in lighting simulation, this simulation running on board the sky simulation device S. For the lighting computation of a room with unilateral facades, one half of the sky hemisphere that the window is facing towards induces as the source of lighting, together with one half of the ground hemisphere with ambient objects. In the light of this, one hemisphere of the sky and ground dome, in sum, faced by the facade of the room is sufficient for luminance monitoring and lighting simulation for a scene with a unilateral facade. Therefore, the sky monitoring device can be positioned in front of the window glazing of the room to perform the monitoring and lighting computation, ensuring the axis of lens aligned in the same direction as that the window glazing is facing towards, as shown in FIG. 3.

(21) FIG. 3 shows a sky monitoring system S in the form of an integrated sky monitoring device mounted outside a building in front of one of the building's window 20. The sky monitoring system S is directed to the sky 21, so as to take pictures of a sky scene, typically including at least parts of the sky and ground dome. In the particular embodiment shown in FIG. 3, the sky 21 is a clear sky and comprises a sun 22.

(22) In a preferred method for using the sky monitoring device S, the RADIANCE program is employed to compute the lighting in an office room, based on back-ward ray tracing algorithm. A scene of an office is first created and then the microprocessor, for example the FPGA chip 3, synthesizes a monitored luminance of the sky and ground half-hemisphere (also referred to as the sky scene) together with the scene of the office room into an octree file, which is taken as an input for lighting computation. Afterwards, the rtrace and rpict, sub-programs of RADIANCE, operate on the HPS part 5 of the FPGA chip 3 to generate the rendering and calculate the illuminance distribution inside the room.

(23) FIG. 4 visualizes the rendering of a reference office on a sunny day, with monitored luminance of the sky, the sun and ambient objects as input. The contour of the surrounding buildings and trees exhibits clearly on the rendered image from a viewpoint inside the room facing towards outside, due to high resolution mapping. As only photometric quantity gets involved, the color of light source is ignored and thus the sky and surrounding landscape appear grey. For tracing 1.41×10.sup.6 rays, it has been discovered that it takes 79.2 seconds to accomplish the rendering on the HPS part 5 of the sky monitoring system's processor, for example the FPGA chip 3, with 1.23 million mapped sky or ground patches as input.

(24) The embedded sky monitoring device according to the disclosure, positioned outside a room and facing towards the sky, can also be used in the lighting estimation of an intra-scene for lighting control. In simple words: for example, the sky monitoring system S shown in FIG. 3 can be used to take pictures of a sky scene outside the room and can calculate a current lighting situation inside the room, for example an illuminance distribution on a workplane inside the room. This illuminance distribution can then be used for lighting control, for example for automatically switching on electric lighting in case the illuminance levels in a certain area of the workplane become too low. This means that the lighting control is being carried out based on the simulation results furnished by the sky monitoring system S and not by actual measurements by means of luxmeters.

(25) Due to its fast speed HDR imaging technique and spectrum response calibration, the embedded sky monitoring device shows its capability in precise detection of solar luminance and tracking of solar position. The sky monitoring device can be used for example for measuring the luminance, azimuth and zenith angle of the sun, which can be potentially applied in the automation of solar PV panels in aligning its angular position according to the sun to maximize its power generation.

(26) The following advantages are at least partially reached by the disclosure: The sky monitoring system S is compact in size and independent of external computation platform such as a personal computer (PC) or servers. The embedded system alone is able to accomplish all the functionality. Wide dynamic range when monitoring the luminance of the direct solar rays and ambient objects in shadow during daytime. High resolution for glare detections and the accuracy of lighting simulation is ensured. Integration of luminance measurement and lighting simulation together. Precise measurement of the angular position of the sun in real-time.

(27) The disclosure has at least the following fields of application: Use as a luminance meter, with high resolution of 10.sup.−7 sr in solid angle, covering high dynamic range including the luminance of the sun (1.6×10.sup.9 cd/m.sup.2) and that of ambient objects (10-100 cd/m.sup.2) at the same time, quick response in seconds. Real-site simulator of lighting of a room for architecture designers. The device could be put at the position of an unconstructed building or in front of the window of a room needed to be refurbished. With real environmental objects and the sky as input, it is able to simulate the lighting conditions for the design of the interior of the building or rooms on real-site. The device can also generate images for users to visualize the lighting condition and help designers to analyze/evaluate the lighting intensity levels (daylighting performance) and whether they meet particular requirements. Application in building control. As a compact and responsive device, it can be installed on the building to monitor the real conditions of the sky and ambient objects and, according to the results of the quasi real-time lighting simulation, it can help control the shading of a building to maintain the illuminance in rooms at a certain level. Real-time control of solar photovoltaic systems, for example to align their position according to the sun position.

(28) The disclosure is not limited to the preferred embodiments described here. The scope of protection is defined by the claims.

(29) Furthermore, the following claims are hereby incorporated into the Detailed Description, where each claim may stand on its own as a separate embodiment. While each claim may stand on its own as a separate embodiment, it is to be noted that—although a dependent claim may refer in the claims to a specific combination with one or more other claims—other embodiments may also include a combination of the dependent claim with the subject matter of each other dependent or independent claim. Such combinations are proposed herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include also features of a claim to any other independent claim even if this claim is not directly made dependent to the independent claim.

(30) It is further to be noted that methods disclosed in the specification or in the claims may be implemented by a device having means for performing each of the respective acts of these methods.

REFERENCE LIST

(31) 1 Image sensor 2 Motor and actuator 3 Field Programmable Gate Array (FPGA) chip 4 FPGA fabric part of the FPGA chip 5 Hard core processor (HPS) part of the FPGA chip 6 Peripheral controller 7 Hardware interface 8 Computation accelerator 9 ARM processing unit 10 SDRAM controller 11 Memory (RAM) 12 Input/output interface 13 Buttons 14 Bridge of HPS to FPGA fabric 15 Bridge of FPGA fabric to SDRAM controller 16 Wide-angle lens 17 Printed circuit board (PCB) 18 SD Memory 19 Outputs 20 Window of a building 21 Sky 22 Sun S Sky monitoring system