Apparatus and method for recording a scene for a plurality of lighting setups using only a portion of each cinematic frame
11258950 · 2022-02-22
Assignee
Inventors
Cpc classification
H04N23/6845
ELECTRICITY
H04N23/74
ELECTRICITY
International classification
Abstract
Apparatus for recording a scene using a plurality of lighting setups in rapid sequence to concurrently record a plurality of motion picture clips of the scene, one motion picture clip for each lighting setup, the plurality of clips together exhibiting negligible motion offset. The apparatus includes a plurality of light sources, a controller to define the plurality of lighting setups using the plurality of light sources and to actuate the lighting setups in sequence, a camera to capture a sequence of micro frames showing the scene illuminated by each one of the plurality of lighting setups in sequence during each micro frame, and optionally a processing module to process the sequence of micro frames to generate a motion picture clip of the scene for each of the lighting setups. The duration of the micro frame sequence is short enough to minimize the need for an algorithm for removing motion artifacts.
Claims
1. An apparatus for recording a scene using a plurality of lighting setups so as to concurrently record a respective plurality of motion picture clips of the scene, one motion picture clip for each lighting setup, the apparatus comprising: a plurality of light sources, each light source configured to illuminate at least part of a scene; a controller configured to: enable a user to define a plurality of lighting setups using the plurality of light sources, and actuate the plurality of lighting setups in accordance with timing signals so as to provide a sequence of lighting setups, the controller configured to actuate the plurality of lighting setups in sequence in accordance with the timing signals, such that a first lighting setup of the plurality of lighting setups is actuated by a timing signal upon a beginning of a macro frame, and a last lighting setup of the plurality of lighting setups is actuated by a timing signal such that the last lighting setup will go dark before an end of the macro frame; and a camera configured to capture a sequence of macro frames, each macro frame being of a duration so as to include a sequence of micro frames, the sequence of micro frames being of a duration of no more than substantially 21 milliseconds, each micro frame initiated by a timing signal, and at least one micro frame being capable of capturing the scene illuminated by one of the plurality of lighting setups.
2. The apparatus of claim 1, wherein the camera is a variable frame rate camera configured to capture micro frames during only a portion of each macro frame, the portion corresponding to the duration of the sequence of micro frames.
3. The apparatus of claim 2, wherein the variable frame rate camera is configured to capture only the sequence of micro frames in each macro frame.
4. The apparatus of claim 2, wherein the variable frame rate camera is configured to capture at least one long micro frame after capturing the sequence of micro frames, the long micro frame being of an extended duration configured to capture light from a continuous light source.
5. The apparatus of claim 1, wherein the camera is a constant frame rate camera configured to record during only a portion of each macro frame, the portion corresponding to the duration of the sequence of micro frames.
6. The apparatus of claim 1, further comprising: a processing module configured to assemble a plurality of motion picture clips, each motion picture clip assembled from a sequence of corresponding micro frames of the sequence of macro frames, each motion picture clip corresponding to one of the lighting setups.
7. The apparatus of claim 1, wherein the timing signals are derived from the camera.
8. The apparatus of claim 1, wherein the timing signals are derived from an external controller.
9. The apparatus of claim 1, wherein the controller is configured to: enable the user to include at least one camera parameter that can change for each micro frame; and sequentially actuate the at least one camera parameter for each of the micro frames in accordance with timing signals.
10. The apparatus of claim 9, wherein the at least one camera parameter can include at least one of: sensitivity (ISO); aperture; ND (neutral density filter); and shutter angle.
11. The apparatus of claim 1, wherein each lighting setup of the plurality of lighting setups is different from other lighting setups in the plurality of lighting setups.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Many additional features and advantages will become apparent to those skilled in the art upon reading the following description, when considered in conjunction with the accompanying drawings, wherein:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
DETAILED DESCRIPTION
(16) With reference to
(17) The controller 104 is also in communication with lighting equipment consisting of: a light source 114, a light source 116, a light source 118, a light source 120, a light source 122, and a light source 124. The light sources are grouped into three lighting setups: a first lighting setup 108 consisting of light source 114 and light source 116, a second lighting setup 110 consisting of light source 118 and light source 120, and a third lighting setup 112 consisting of light source 122 and light source 124.
(18) Each light source that is capable of providing strobed light can be one or more LEDs, or a Xenon strobe light, or other light source that can be switched on and off relatively rapidly, i.e., any light source that can exhibit strobe light behavior.
(19) Flash duration of a light source is commonly described by two numbers that are expressed in fractions of a second: t.1 is the length of time the light intensity is above 0.1 (10%) of the peak intensity t.5 is the length of time the light intensity is above 0.5 (50%) of the peak intensity
(20) For example, a single flash event might have a t.5 value of 1/1200 and a t.1 value of 1/450. These values determine the ability of a flash to “freeze” moving subjects in applications such as sports photography.
(21) Individual strobe flashes typically last approximately 200 microseconds, i.e., 0.2 milliseconds, i.e., ⅕ of a millisecond, but can be sustained for greater or lesser periods of time, depending on the strobe's intended use.
(22) Here are some time durations for comparison: 0.2 milliseconds—duration of a typical strobe flash 1 millisecond (1 ms)—duration of a typical photo flash. 2 milliseconds to 5 milliseconds—typical response time in LCD computer monitors, especially high-end displays 8 milliseconds— 1/125 of a second, a standard still camera shutter speed (125) 16.68 milliseconds (1/59.94 second)—the amount of time one field lasts in 29.97 fps interlaced video (commonly but erroneously referred to as 30 fps) 33.367 milliseconds—the amount of time one frame lasts in 29.97 fps video (most common for NTSC-legacy formats) 41.667 milliseconds—the amount of time one frame lasts in 24 fps video (most common cinematic frame rate) 41.708 milliseconds—the amount of time one frame lasts in 23.976 fps video (cinematic frame rate for NTSC-legacy formats) 134 milliseconds—the time taken by light to travel around the Earth's equator 200 milliseconds—the time it takes the human brain to recognize emotion in facial expressions 300 to 400 milliseconds—the time for the human eye to blink 1000 milliseconds—the time for one second to pass.
(23) The camera 106 records the action in scene 102 as a sequence of macro frames 126. Each macro frame 132 of the sequence 126 can be a cinematic frame with a duration of 41.6667 milliseconds, or 1/24.sup.th of a second. However, each macro frame 132 may be of a longer or shorter duration, depending on the desired cinematic film rate.
(24) In some embodiments, the camera can be a high-speed variable frame rate camera. The high-speed variable frame rate camera can be configured to record during only a portion 128 of each macro frame 132, the portion 128 corresponding to the duration of the sequence of micro frames A, B, C relative to the total duration 128 and 130 of the macro frame. The high-speed variable frame rate camera can also be configured to not record during the remainder 130 of the duration 132 of each macro frame. Alternatively, the high-speed variable frame rate camera can be programmed to define at least one long frame as a “throw away” frame, to define a time period during the macro frame where the image data is not needed for post-processing. The long frame can also be used to capture continuous light from a continuous light source, as will be explained with reference to
(25) The high-speed variable frame rate camera can be programmed to precisely change the frame rate within the duration of each macro frame. For example, a high-speed variable frame rate camera can be programmed to record a 400 microsecond micro frame duration, followed by a much longer 40,000 microsecond long frame duration within the same macro frame, as shown in
(26) Some examples of high-speed variable frame rate cameras are in the Vision Research Phantom V® series of cameras, featuring Burst Mode Acquisition, which are capable of capturing a sequence of micro frames in a short duration burst, each burst being triggered within each much longer duration macro frame.
(27) In some embodiments, the camera can also be a high-speed constant frame rate camera, if the high-speed constant frame rate camera can be configured to record during only a portion of each macro frame, the portion corresponding to the duration of the sequence of micro frames relative to the total duration of the macro frame. The high-speed constant frame rate camera can be configured to not record during the remainder of the duration of each macro frame after the burst of micro frames.
(28) As shown in
(29) In this embodiment, every macro frame 132 of the sequence of macro frames 126 includes a sequence of three micro frames 128 and one long frame 130.
(30) In other embodiments, the number and/or position of the micro frames can be different. For example, in another embodiment, each macro frame can include one long frame followed by a sequence of six micro frames that are of short duration (not shown).
(31) Under control of the controller 104, the scene 102 is sequentially illuminated by the first lighting setup 108, then the second lighting setup 110, and then the third lighting setup 112. The first micro frames A 134 record the scene 102 illuminated by the first lighting setup 108, the second micro frames B 136 record the scene 102 illuminated by the second lighting setup 110, and the third micro frames C 138 record the scene 102 illuminated by the third lighting setup 112.
(32) Lighting intervals A, B, and C are shown indicating the duration of the three micro frame time intervals within the sequence 128 of three micro frames, lighting intervals A, B, and C corresponding to: the first micro frames 134, the second micro frames 136, and the third micro frames 138, respectively.
(33) The controller 104 controls the timing of the lighting intervals A, B, and C within the sequence of the three micro frames 128, and the timing of the long frame 130 (1, 2, 3, . . . n). The lighting intervals A, B, and C are chosen to be in sequence, and short in duration, such that the illumination from the first lighting setup 108, the illumination from second lighting setup 110, and the illumination from the third lighting setup 112 provide minimal motion offset between micro frames corresponding to the three lighting setups.
(34) Each macro frame 132 of the macro frame sequence 126 includes three micro frames 128 corresponding to each of the three lighting setups. The first micro frames 134 (corresponding to the A's) correspond to the first lighting setup 108, the second micro frames 136 (corresponding to the B's) correspond to the second lighting setup 110, and the third micro frames 138 (corresponding to the C's) correspond to the third lighting setup 112.
(35) In some embodiments, the controller 104 is configured to enable the user to include at least one camera parameter that can change for each micro frame, and sequentially actuate the at least one camera parameter for each of the micro frames in accordance with timing signals derived from a micro frame rate of the camera 106. For example, the at least one camera parameter can include at least one of: sensitivity (ISO); aperture; ND (neutral density filter); and shutter angle.
(36) One of average skill in the art will know that it is also possible to replace the controller 104 with a plurality of dedicated controllers, each controller dedicated to controlling a single light, or a single lighting set up. Alternatively, a controller could be built into each light. In these embodiments, each dedicated controller can execute software to control a specific light or lighting set up, and the camera could provide control signals to actuate each dedicated controller.
(37) With reference to
(38) Also included is the controller 104 in communication with the camera 106, which is configured to record the scene 102. The controller 104 synchronizes operation of the camera 106 with activation of the lighting setups 202, 204, and 206, each lighting setup providing short duration illumination for the scene 102.
(39) In this embodiment, a continuous light source 214 provides constant illumination for the scene 102.
(40) With reference to
(41) In this embodiment, the signal voltage alternates between the two binary states of 5.0 volts and 0.0 volts, for example, to provide low trigger signals 316, 318, and 320. In this embodiment, the low trigger signals 316, 318, 320 come from the camera in accordance with a micro frame rate of the camera, and are provided to the controller 104 to activate in sequence seven different lighting setups, for example.
(42) In some embodiments, an external timing source sends a timing signal to the camera.
(43) In other embodiments, an external timing source sends a timing signal to both the camera and to a light controller, or directly to a plurality of lighting setups.
(44) A low-going transition of 5V to 0V represents a trigger signal. Examples of these are the micro frame low trigger signals 316 that start each of the five micro frames A, B, C, D, E, and the long frame low trigger signals 318 and 320. In other embodiments, a high-going transition from 0V to 5V represents a trigger signal.
(45) In this embodiment, the first low trigger signal 316 starts the first micro frame A 310, and the sixth low trigger signal 318 starts the first long frame F, followed by the seventh low trigger signal 320 that starts the second long frame G. The long frames F and G together form the long interval 304. One or both long frames F and G can be used to capture light from a scene illuminated by a continuous light source, such as the continuous light source 214 of
(46) Regarding F and G, after the short micro frames A, B, C, D, E 316 there are long micro frames F and G which occupy the remaining portion 304 of the macro frame 306. These additional frames F and G can be recorded, or not. When recorded, the long frames F and/or G are typically used to record ambient exposure from the continuous light source(s) 214, or any other continuous light source, including daylight, for example.
(47) The duration of F or G 304 of the macro frame 306 is the product of the macro frame duration 306 and a shutter angle. In this context, the shutter angle is defined as the ratio of the duration of F or G of the macro frame 304 to the duration of the entire macro frame 306. For example, if macro frame 306 is 1/24th of a second, and the shutter angle is 180 degrees (which is half of the overall frame 306), F or G is 1/48th of a second.
(48) Two possible cases with long frames F and G, for example:
(49) Case 1:
(50) Duration of micro frames 302+Duration of F=(Macro Frame Duration 306)/2
(51) Duration of G=(Macro Frame Duration 306)/2
(52) Case 2:
(53) Duration of micro frames 302+Duration of G=(Macro Frame Duration 306)/2
(54) Duration of F=(Macro Frame Duration 306)/2
(55) One possible case with a single long frame F only, for example:
(56) Duration of micro frames 302+Duration of F=Macro Frame Duration 306
(57) Thus, there can be a long frame F wherein ambient continuous light is captured, and there can be a long frame G wherein ambient continuous light is captured. It is also possible that there are no long frames F or G, and the camera does not record light during 304, only during 302. It is also possible for ambient continuous light to be captured during a long frame F, and there would be no long frame G.
(58) This embodiment includes five micro frame low trigger signals A, B, C, D, E 316 that the camera 106 sends to the controller 104 (shown in
(59) Also shown are the long frame low trigger signals 318 and 320, which mark the start of the optional frames F and G of the remaining portion 304, during which continuous lighting can be recorded, such as the continuous lighting starting at the long frame low trigger signal 318 and ending at the long frame low trigger signal 320.
(60) The micro frame low trigger signals 316 and the long frame low trigger signals 318 and 320 can be short in duration, such as 1 microsecond, for example. Alternatively, each trigger signal can be the duration of the respective micro frame or of the respective long frame exposure duration.
(61) In some embodiments a high-speed variable frame rate camera can be programmed to define the remaining portion 304 as a “throw away” frame, used to exclude from recording many frames during the macro frame 306 where that image data is not needed for post-processing. This partial frame recording greatly reduces image data storage requirements (versus storing high speed image data and continuous illumination data throughout the entire duration 306 of the macro frame).
(62) Typically, a macro frame 306 includes one to ten micro frames 302. The most useful range is from one to thirty micro frames per macro frame.
(63) In some embodiments, the length of each of the micro frames A, B, C, D, and E is 400 microseconds, and the length of the remaining portion 304 is substantially 39,667 microseconds, corresponding to a cinematic frame rate of 1/24.sup.th of a second, or a total macro frame duration of 41,666.67 microseconds. For duration examples, see
(64) The micro frames A, B, C, D, and E are chosen to be in sequence, and to be of short duration so as to provide minimal motion offset among the images captured within the sequence of five micro frames 302.
(65) Each macro frame 306 is recorded in sequence to form a sequence of macro frames 322.
(66) The camera 106 (shown in
(67) In some embodiments, the duration of the sequence of five micro frames 302 is less than 5 milliseconds. In the case of a sequence of 1 to ten micro frames, the duration of such a sequence is typically 0.2 to 10 milliseconds. In the case of a sequence of thirty micro frames, the duration of such a sequence is typically less than 30 milliseconds.
(68) The shortness of the duration of the sequence of five micro frames 302 is chosen to reduce motion artifacts, determined by the motion characteristics of the subject being filmed. The more micro frames in the sequence, the more likely motion blur and motion offset will be introduced.
(69) In some embodiments, the controller 104 is configured to actuate the plurality of lighting setups (e.g., 108, 110, 112) in sequence in accordance with the timing signals provided by the camera 106, such that a first lighting setup 108 of the plurality of lighting setups is actuated by a timing signal after a beginning of a macro frame, and a last lighting setup 112 of the plurality of lighting setups is actuated by a timing signal such that the last lighting setup will go dark before an end of the macro frame.
(70) In some embodiments, the timing signals for the micro frames and the long frames are derived from the camera 106, where the timing signals for the short and long micro frames are set in the camera, and provided by the camera 106 to the controller 104, which in turn controls the lighting setups 108, 110, 112, for example.
(71) With reference to
(72) Each macro frame 132 includes a sequence of three micro frames A B C 128 and one long frame (1, 2, 3), the micro frames A B C 128 corresponding to short duration lighting intervals that substantially minimize visual artifacts (such as motion offset between the micro frames A, B, and C corresponding to the three lighting setups) when post processing the sequence of macro frames 126, if slow-speed motion is present in the scene 102.
(73) With reference to
(74) Each macro frame 410 includes a sequence of three micro frames A B C 408 and one long frame (1, 2, 3), the micro frames A B C corresponding to short duration lighting intervals that substantially minimize visual artifacts (such as motion offset between the micro frames A, B, and C corresponding to the three lighting setups) when post processing the sequence of macro frames 402, if standard-speed motion is present in the scene 102.
(75) With reference to
(76) Each macro frame 414 includes a sequence of three micro frames A B C 412 and one long frame (1, 2, 3), the micro frames A B C corresponding to short duration lighting intervals that substantially minimize visual artifacts (such as motion offset between the micro frames A, B, and C corresponding to the three lighting setups) when post processing the sequence of macro frames 404, if slow-speed motion is present in the scene 102.
(77) With reference to
(78) Each macro frame 418 includes a sequence of three micro frames A B C 416 and one long frame (1, 2, 3), the micro frames A B C corresponding to short duration lighting intervals that substantially minimize visual artifacts (such as motion offset between the micro frames A, B, and C corresponding to the three lighting setups) when post processing the sequence of macro frames 406, if standard-speed motion is present in the scene 102.
(79) With reference to
(80) The sequence of macro frames 126 produce three respective pieces of footage of the scene 102 (shown in
(81) Three respective clips 502, 504, 506 of footage of the scene are produced, each clip of footage showing the scene illuminated by one of the three lighting setups.
(82) With reference to
(83) The controller 104 activates the first lighting setup 602 at a time within the “A” micro frames of the sequence of frames 126 (shown in
(84) The first lighting setup 602 includes the light source 608 and the light source 610, both synchronized by the controller 104 to provide short duration illumination to the scene 102 at the same time, corresponding to the “A” micro frames in the sequence of frames 126 (shown in
(85) The second lighting setup 604 includes the light source 612 and the light source 614, both synchronized by the controller 104 to provide short duration illumination to the scene 102 at the same time, corresponding to the “B” micro frames in the sequence of frames 126 (shown in
(86) The third lighting setup 606 includes the light source 616 and the light source 618, both synchronized by the controller 104 to provide short duration illumination to the scene 102 at the same time, corresponding to the “C” micro frames in the sequence of frames 126 (shown in
(87) Each light source is controlled by the controller 104 that receives timing signals from the camera 106 recording the scene so as to provide sequential activations to the first lighting setup 602, the second lighting setup 604, and the third lighting setup 606.
(88) With reference to
(89) The controller 104 activates the first lighting setup 702 at a time within the “A” micro frames of the sequence of frames 126 (shown in
(90) The first lighting setup 702 includes the light source 708, synchronized by the controller 104 to provide short duration illumination to the scene 102, corresponding to the “A” micro frames in the sequence of frames 126 (shown in
(91) The second lighting setup 704 includes the light source 710, the light source 712 and the light source 714, all three synchronized by the controller 104 to provide short duration illumination to the scene 102 at the same time, corresponding to the “B” micro frames in the sequence of frames 126 (shown in
(92) The third lighting setup 706 includes the light source 716 and the light source 718, both synchronized by the controller 104 to provide short duration illumination to the scene 102 at the same time, corresponding to the “C” micro frames in the sequence of frames 126 (shown in
(93) Each light source is controlled by the controller 104 that receives timing signals from the camera 106 recording the scene so as to provide sequential activations to the first lighting setup 702, the second lighting setup 704, and the third lighting setup 706.
(94) With reference to
(95) The controller 104 activates the first lighting setup 802 at a time within the “A” micro frames of the sequence of frames 126 (shown in
(96) The first lighting setup 802 includes the light source 808 and the light source 810, both synchronized by the controller 104 to provide short duration illumination to the scene 102 at the same time, corresponding to the “A” micro frames in the sequence of frames 126 (shown in
(97) The second lighting setup 804 includes the light source 812, the light source 814 and the light source 816, all three synchronized by the controller 104 to together provide short duration illumination to the scene 102 at the same time, corresponding to the “B” micro frames in the sequence of frames 126 (shown in
(98) The third lighting setup 806 includes the light source 818, synchronized by the controller 104 to provide short duration illumination to the scene 102, corresponding to the “C” micro frames in the sequence of frames 126 (shown in
(99) Each light source is controlled by the controller 104 that receives timing signals from the camera 106 recording the scene so as to provide sequential activations to the first lighting setup 802, the second lighting setup 804, and the third lighting setup 806.
(100) With reference to
(101) The method includes selecting 902 a plurality of light sources, with each light source configured to illuminate the scene 102 (shown in
(102) Next, the method 900 includes selecting 904 a plurality of lighting setups from the plurality of light sources, and then actuating 906 the plurality of lighting setups in sequence from the plurality of light sources. Each lighting setup can be selected so as to be different from the other lighting setups, or two lighting setups can share one or more light sources in common.
(103) The method 900 next includes capturing 908 a sequence of macro frames, each macro frame including a sequence of micro frames, in accordance with timing signals, including actuating each lighting setup corresponding to each micro frame, in the sequence of micro frames within each macro frame.
(104) Optionally, the method 900 may also include processing 910 the sequence of macro frames to generate a plurality of motion picture clips corresponding to the plurality of lighting setups.
(105) 1/60.sup.th of a second is a common threshold whereby a number of everyday types of motions appear frozen at this recording speed or faster. At a frame rate of 60 FPS, each frame has a duration of 1/60.sup.th of a second, equal to 16.6667 milliseconds.
(106) Referring to
(107) With reference to
(108) Instead of using the internal settings of the camera 106, an external controller 1120 can serve as a signal generator to provide a signal to the camera 106 which determines the timings of the macro and micro frames. One example of this is a waveform generator (such as the Keysight™ 33511B) which, when attached with a BNC cable, to the F-Sync connector on a high speed camera such as the Vision Research™ v2460, can supply the necessary signal. In the example of the Keysight™ 33511B, the signal is a 5V signal which drops to 0V for at least one microsecond to close out the previous frame and to cause a new frame to start. The signal then returns to 5V.
(109) For example,
(110) For example, to run the camera for a 10 second shot, the signal in
(111) In other embodiments, the external controller 1120 can be a ‘custom controller’ or other device generating timing signals, including the F-sync type outlined above, and others commonly used in the industry such as Genlock™ (Tri Level, BiLevel), FrameSync™, Strobe Signal, or RIG. In addition to sending signals directly to the camera, 1120 can also be connected directly to the main system controller 104 (shown as a dashed line between 1120 and 104 in
(112) Thus, in some embodiments, the timing signals are derived from the external controller 1120 that controls the camera 106, and the camera 106 controls the lighting setups 108, 110, 112.
(113) This can be done in a variety of ways which involve the system controller 104 and/or the external controller 1120:
(114) An external controller 1120 generates signals for the micro frames and the long frames which are sent to the camera 106, which sends them on to the system controller 104.
(115) An external controller 1120 generates signals for the micro frames and the long frames which are sent both to the camera 106 and to the system controller 104.
(116) An external controller 1120 generates signals for the micro frames and the long frames which are sent to the camera 106, and in unison with these timing signals it also signals the lights to fire. Thus, there is only one controller (external controller 1120) which sends signals directly to the camera and to the lights.
(117) An external controller 1120 generates signals for the micro frames and the long frames which are sent to the camera 106. The camera 106 sends a signal back to the external controller 1120, which then uses that signal as the basis for sending signals to the lights to fire. Thus, there is only one controller (external controller 1120) which sends signals directly to the camera and to the lights.
(118) In some embodiments, the lights receive signals from the external controller 1120 or the controller 104 to trigger at a FPS (frame rate) that is a multiple of the FPS (frame rate) at which the camera 106 is recording. Thus, the lights can operate at a firing rate which is a multiple of the camera frame rate. For example the camera 106 can run at 24 FPS, while the lights are still synced as explained above, but run at a rate of 96 FPS.
(119) Other modifications and implementations will occur to those skilled in the art without departing from the spirit and the scope of the invention as claimed. Accordingly, the above description is not intended to limit the invention, except as indicated in the following claims.