Cloaking systems and methods
10739111 ยท 2020-08-11
Assignee
Inventors
Cpc classification
H04N13/232
ELECTRICITY
H04N5/74
ELECTRICITY
H04N13/229
ELECTRICITY
G01C3/14
PHYSICS
G02B2207/123
PHYSICS
G01S17/42
PHYSICS
F41H3/00
MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
H04N13/307
ELECTRICITY
G02B27/0075
PHYSICS
G02B27/0081
PHYSICS
H04N13/305
ELECTRICITY
International classification
F41H3/00
MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
H04N9/31
ELECTRICITY
H04N13/307
ELECTRICITY
H04N13/232
ELECTRICITY
H04N13/305
ELECTRICITY
G02B27/00
PHYSICS
H04N13/229
ELECTRICITY
H04N5/74
ELECTRICITY
Abstract
Digital cloaking is a method for practical cloaking, where space, angle, spectrum and phase are discretized. At the sacrifice of spatial resolution, a good approximation to an ideal cloak can be achieveda cloak that is omnidirectional, broadband, and operational for the visible spectrum, three-dimensional (3D), and phase-matching for the light field, among other attributes. One example of a digital cloak is an active cloak that uses lenticular lenses, similar to integral imaging for 3D displays. With the continuing improvement in commercial digital technology, the resolution limitations of a digital cloak may be minimized, and a wearable cloak can be implemented.
Claims
1. A cloaking system, comprising: an electronic light detector configured to detect light approaching a cloaking volume from a first side, the electronically detected light characterized by a plurality of first light rays incident on the light detector; an electronic light emitter configured to emit light away from a second side of the cloaking volume, the electronically emitted light characterized by a plurality of second light rays, the light emitter comprising a first lens array positioned over an emitter array; wherein individual regions of the detector are each mapped to individual regions of the emitter such that detection of light at one of the individual regions of the detector results in emission of light at one of the individual regions of the emitter; wherein a first region of the detector located at a longitudinal position z.sub.i and configured to detect a light ray characterized by a transverse position y.sub.i and an angle .sub.i is mapped to a first region of the emitter located at a longitudinal position z.sub.f and configured to emit a light ray characterized by a transverse position y.sub.f and an angle .sub.f such that:
2. The cloaking system of claim 1, wherein the electronic light detector comprises a second lens array positioned over a detector array.
3. The cloaking system of claim 2, wherein the electronic light detector is configured to detect both positions and directions of at least some of the plurality of first light rays incident on the light detector.
4. The cloaking system of claim 2, wherein the electronic emitter array comprises a plurality of light emission regions, wherein at least some of the lenses of the first lens array are each positioned over a group of the plurality of light emission regions, wherein the electronic detector array comprises a plurality of light detection regions, wherein at least some of the lenses of the second lens array are each positioned over a group of the plurality of light detection regions.
5. The cloaking system of claim 4, wherein the first light detection and emission regions are aligned along a first axis, and a second light detection region corresponding to a second light emission region are aligned along a second axis, wherein the first and second axes are non-parallel.
6. The cloaking system of claim 5, wherein the light detection and emission regions include a third light detection region corresponding to a third light emission region, the third light detection and emission regions aligned along a third axis, wherein the first and third axes are non-parallel, wherein the second and third axes are parallel.
7. The cloaking system of claim 2, wherein the detector array comprises at least one solid state image sensor.
8. The cloaking system of claim 7, wherein the solid state image sensor comprises a CCD sensor or a CMOS sensor.
9. The cloaking system of claim 2, wherein the detector and emission arrays comprise fiber optics.
10. The cloaking system of claim 2, wherein the cloaking volume is between the detector and emitter arrays, wherein the detector and emitter arrays are between the first and second lens arrays.
11. The cloaking system of claim 1, wherein the cloaking system is configured to at least partially surround the cloaking volume.
12. The cloaking system of claim 11, wherein the electronic light detector and electronic light emitter are planar.
13. The cloaking system of claim 11, wherein at least one of the electronic light detector and electronic light emitter are curved or angular.
14. The cloaking system of claim 13, wherein the electronic light detector and electronic light emitter each include at least one edge, wherein the edge of the electronic light detector meets the edge of the electronic light emitter.
15. A method of cloaking a cloaking volume that is at least partially between a light detector and light emitter, the method comprising: measuring positions and directions of at least some of a first plurality of light rays approaching the light detector, the light detector comprising a first lens array configured to measure the positions of the light rays and a detector array configured to measure the directions of the light rays; and emitting a second plurality of light rays from the light emitter based on the measured positions and directions of the first plurality of light rays, the light emitter comprising a second lens array and an emitter array, wherein at least some of the second plurality of light rays are substantially aligned in positions and directions with at least some of the first plurality of light rays, such that the cloaking region is at least partially cloaked to human vision; wherein individual regions of the light detector are each mapped to individual regions of the light emitter such that detection of light at one of the individual regions of the light detector results in emission of light at one of the individual regions of the light emitter; wherein a first region of the light detector located at a longitudinal position z.sub.i and configured to detect a light ray characterized by a transverse position y.sub.i and an angle .sub.i is mapped to a first region of the light emitter located at a longitudinal position z.sub.f and configured to emit a light ray characterized by a transverse position y.sub.f and an angle .sub.f such that:
16. A multi-directional cloaking system, comprising: a plurality of discrete electronic light detection pixels configured to detect light approaching a cloaking volume, the detected light characterized by a plurality of first light rays; a plurality of discrete electronic light emission pixels configured to emit light away from the cloaking volume, the emitted light characterized by a plurality of second light rays; the plurality of discrete electronic light emission pixels configured to emit the emitted light such that at least some of the second light rays are substantially aligned with at least some of the first light rays; wherein individual light detection pixels are each mapped to individual light emission pixels such that detection of light at one of the individual light detection pixels results in emission of light at one of the individual light emission pixels; wherein a first light detection pixel located at a longitudinal position z.sub.i and configured to detect a light ray characterized by a transverse position y.sub.i and an angle .sub.i is mapped to a first light emission pixel located at a longitudinal position z.sub.f and configured to emit a light ray characterized by a transverse position y.sub.f and an angle .sub.f such that:
17. The multi-directional cloaking system of claim 16, wherein the multi-directional cloaking system is spherically symmetric.
18. The multi-directional cloaking system of claim 16, wherein at least some of the plurality of discrete electronic light emission pixels are spatially discrete, and wherein at least some of the plurality of discrete electronic light emission pixels each comprise a plurality of electronic discrete light emission sub-pixels corresponding to a plurality of discrete light emission directions.
19. The multi-directional cloaking system of claim 16, wherein at least some of the plurality of discrete electronic light detection pixels are spatially discrete, and wherein at least some of the plurality of discrete electronic light detection pixels each comprise a plurality of discrete light detection sub-pixels corresponding to a plurality of discrete light detection directions.
20. The multi-directional cloaking system of claim 19, wherein at least some of the plurality of discrete electronic light detection pixels comprise apertures; wherein at least some of the plurality of discrete light detection sub-pixels comprise non-planar arrays of light detectors beneath at least some of the apertures.
21. The multi-directional cloaking system of claim 19, wherein at least some of the plurality of discrete electronic light detection pixels comprise lenslets; wherein at least some of the plurality of discrete light detection sub-pixels comprise light detectors beneath at least some of the lenslets.
22. The multi-directional cloaking system of claim 16, wherein the mapping between the discrete light electronic detection pixels and discrete light electronic emission pixels is dynamic, such that movement of one or more of the discrete electronic light detection pixels relative to the discrete electronic light emission pixels changes the mapping between particular discrete electronic light emission pixels and electronic light detection pixels.
23. The multi-directional cloaking system of claim 16, wherein the multi-directional cloaking system is phase-matching.
24. A passive cloaking system comprising: an input lenslet array comprising a plurality of lenses for collecting light rays with direction and position properties; an output lenslet array comprising a plurality of lenses for emitting the collected light rays; wherein the input lenslet array and the output lenslet array define a cloaking region between the input lenslet array and the output lenslet array; and a plurality of optical fibers routed through the cloaking region and optically coupling lenses of the input lenslet array with lenses of the output lenslet array such that the light rays collected by the input lenslet array are emitted by the output lenslet array and such that at least some of the emitted light rays are substantially aligned with at least some of the collected light rays; wherein individual input lenses are each mapped to individual output lenses such that collection of light at one of the individual input lenses results in emission of light at one of the individual output lenses; wherein a first input lens located at a longitudinal position z.sub.i and configured to collect a light ray characterized by a transverse position y.sub.i and an angle .sub.i is mapped to a first output lens located at a longitudinal position z.sub.f and configured to emit the collected light ray at a transverse position y.sub.f and an angle .sub.f such that:
25. The passive cloaking system of claim 24, wherein the emitted light rays preserve spectral properties of the collected light rays.
26. The passive cloaking system of claim 25, wherein the lenses of the input lenslet array focus collected light into a fiber core of a corresponding optical fiber.
27. The passive cloaking system of claim 26, wherein the plurality of optical fibers comprise multi-mode optical fibers.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
DETAILED DESCRIPTION
(23)
(24) In some instances, it may be desirable to minimize or eliminate this effect. In other words, in some instances cloaking systems or methods may make objects behind the cloaking region appear to be where they are and/or act the same way as if the system was filled with the surrounding medium (e.g., to cloak in air, the cloaking device may act as if it was filled with air).
(25) Ray-Preserving Cloaking Systems
(26)
(27) The light detector 202 is configured to detect light approaching the cloaking region 204, which may be characterized by incoming light rays such as rays 208, 210. In this embodiment, the light detector 202 is configured to detect the positions and directions of the incoming light rays 208, 210. In some embodiments, the light detector 202 is also configured to detect the spectral content of incoming light rays 208, 210. The light detector 202 includes a lens array 212 (e.g. a micro-lens array) positioned over a detector array 214 (e.g. an image sensor). As shown in
(28) In the embodiment of
(29) The light emitter 206 of
(30) The light emitter 206 includes a lens array 216 with an emitter array 218 below it. The emitter array 218 includes several light emission regions (e.g. individual pixels) beneath each lens of the lens array 216. Particular light emission regions in the emitter array 218 correspond to particular light detection regions in the detector array 214. For example, in some embodiments, every pixel in the detector array 214 may have a single emission pixel in the emitter array 218. The corresponding pixels of the emitter array 218 can then (in conjunction with lens array 216) emit outgoing light rays 220, 222 with both the proper direction up to the direction resolution of the detector array 214 and the proper position up to the position resolution of the lens array 212. In some embodiments, the light detector 202 and/or light emitter 206 may be configured to also preserve or otherwise account for the frequency and other aspects of incoming light so that the emitter characteristics match the incoming light characteristics. Thus, in the embodiment of
(31) Those of skill in the art will recognize that a wide variety of lens arrays may be used for the cloaking system, including a wide variety of commercially available microlens arrays formed of various substrates, in various lens or lenslet pitches, various lens shapes, various fill factors, various lens diameters, and other attributes. In some embodiments, the lens arrays may have a lens pitch in the range of 50 m to 5 mm. In some embodiments, the lens arrays may have collection/emission angles in the range of 10 to 60 degrees, typical of 3D and animation lenticular lenses. Larger angles can be achieved through careful lens processing. For embodiments that allow for large angle cloaking, lens arrays incorporating aspheres may be utilized, although those of skill in the art will recognize that larger angle cloaking applications may, in at least some instances, also require larger number of light detection/emission regions for each lens. Typical lenticular and Fly's eye lenses come in a large range of resolution. For example, commercially available lenticular lenses have up to more than 100 lenses per inch and down to as little as 7.5 lenses per inch.
(32) A wide variety of detector/emitter arrays may also be used for the cloaking system. In some instances, the detector array may be one or more solid state image sensors, such as one or more CCD or CMOS sensors. In some instances, the emitter array may be a CRT, LCD, LED, plasma, or other type of display or monitor, or include similar components as such displays or monitors.
(33) Returning to cloaking device of the embodiment of
(34) In some embodiments, the correlation between particular detection and emission regions of the arrays 214 and 218 is fixed. For example, in the embodiment shown in
(35) While not shown in the Figures, in some cloaking system embodiments, internal components of the system (e.g. components behind the light detector 202 and light emitter 206) may be configured to visually reconstruct the external environment. For example, a display inside of the cloaking system may display what is on the outside of the cloak, using the same ray information collected by the light detector 202.
(36)
(37) In some non-limiting instances, a cloaking system may be configured to cloak a cloaking volume from multiple sides of the cloaking volume. For example, a first side of the cloaking volume could include detection and emission arrays, and a second side of the cloaking volume could also include detection and emission arrays, with the emission array on the second side emitting light in response to light detected by the detector array on the first side, and the emission array on the first side emitting light in response to light detected by the detector array on the second side.
(38) Passive Ray-Preserving Cloaking Systems
(39) While cloaking systems such as those described above may fool the human visual system, in at least some instances, both hyperspectral imaging and ranging detection could reveal the cloaking system. For a lidar system, a source emits a pulse or frequency ramped signal. In 3D lidar, the time-of-flight for each transverse pixel may reveal a flat region at an unexpected distance and thus may reveal the cloak. In hyperspectral imaging, images are reconstructed every few nm over the range of frequencies of interest. For a standard Red-Green-Blue flat panel display (the emission plane), the artificial color could potentially be apparent when reconstructed.
(40) In some instances, a passive cloaking system may be employed to overcome these advanced detection techniques. In some embodiments, the passive cloaking system may use fibers between the correlated detection and emission regions (e.g. with the two ends of a fiber constituting the correlated detection and emission pixels). For example, all of the incoming light may be collected into broadband multi-mode fibers and routed to and emitted from the appropriate emission regions to preserve the rays, including their spectral properties along with their direction and position properties. With high quality anti-reflection coatings, nearly 100% of the incoming light can be collected by layering microlens arrays of appropriate size. On the collection end, a low resolution lens array would collect the position information and a high resolution array would ensure all of the light was focused into the fiber cores. This would ensure that the cloak was high brightness.
(41) In some instances, such a passive cloaking system may counter a hyperspectral imaging system, because the natural light would be collected and appropriately reemitted with nearly 100% efficiency. Optical light detection and ranging systems could detect a shift in depth from a reference background, but perhaps small enough to be unnoticed when the background is not smooth. In some instances, such a cloaking system could be further modified to reduce its radar cross-section, such as by creating bends in the material to reflect radar signals downward or upward.
(42) Background Projection
(43) In some instances, background projection may be used as an alternative to ray-preserving cloaks.
(44) To understand the basic physics of background projection, consider
(45)
(46) The cloaking system shown in
(47) In the
(48) Thus, in the system of
(49) In at least some embodiments, even though the projected background is made up of columned sub-images projected in different directions, the projected background will not appear to be dimmer than the actual background. When one looks at a standard projected image, the light is scattering from an emission pixel in a steradians. However, in at least some embodiments of the present invention, the light for a projected image under a lens arrays scatters in only in the desired direction and thus increases the flux to the observer. The higher the resolution of the emission array or the lower the pitch of the lenticular array, the more unique projections of the background there will be that can be observed on the cloak.
(50) To demonstrate one non-limiting example of a background projection system, a 28-view lenticular lens array (10 lenses per inch) on top of an HD resolution 25 HP Pavilion 25bw monitor was used. The lenticular lenses were angled at 18.43 degrees relative to the vertical (3 to 1 slope). This allowed for sub-pixel interlacing (each color of the pixel is part of the interlace) and eliminates many of the fringing effects that can happen with pixel interlacing. The monitor had 1920 pixels on the horizontal with a 21.8 inch horizontal active area. Multiplying by 3 colors and dividing by Cos(ArcTan(1/3)) (owing to the slant to the lenses) and the number of lenses per inch yields 27.85 slanted pixels per lens or 28 views. The background image was loaded into a Mathematica program developed for creating sub-pixel interlacing. The horizontal RGB pixel values of the image were appropriately shifted upward by raising every red value up two spatial pixels, every green value up one spatial pixel and keeping the blue value fixed. The 28 views of the background were then placed on each of the diagonal pixels created within the lenses. Each view along each of the 28 diagonals then presents the image associated with a particular angle. It can be seen in
(51) Static Cloaking Systems
(52) In some instances, the cloaking system may take the form of a static cloaking system. In one such example, collection optics may be used to acquire a scene, which may be printed in high resolution and applied to the back of a lens array (e.g. a lenticular array). A standard Epson printer, for example, can print images with up to 5760 dots per inch. Using a 60 lens per inch array, one can then have 96 unique directions for each lens giving both sub-mm resolution and near continuous changes of the scene as the observer moves relative to the cloaking system and background. In the case of a static system, one may use Integral Imaging, and, in some instances, may match the direction of the rays to the background as the observer moves relative to the cloaking system. In such an embodiment, the acquisition may be done where the emitter will be placed. In one example, the acquired scene may include several images captured from different positions or angles relative to a background to be used for the cloaking system.
(53) To demonstrate an example of a static cloaking device according to the design outlined above, a sliding camera may be used to take video at 30 frames per second to acquire the scene. A lenticular lens array with 40 LPI and 18 views corresponding to 720 dots per inch from an Epson printer was used. To demonstrate the power of the technique in acquiring the image at one position and projecting it at another position, the image was acquired 16 inches away from the books, but the projection is 4 inches away while preserving the spatial and proper physics of relative motion for the background objects. By choosing the correct scan speed for the camera, one image per 40.sup.th of an inch was collected. The 40 lpi 3D lenses have a viewing angle of approximately 18 degrees. It was determined that 18 degrees corresponded to approximately 500 pixels of the camera images. To project at a different distance than acquisition distance, 18 columns of pixels were chosen from every camera image and each column was interlaced under a unique lens. Each of those columns corresponded to a unique or about 1 degree of angular separation. This meant that every 28.sup.th column of pixels ( 1/18.sup.th of 500 pixel columns) was used as a view and for this demonstration was placed 8 pixels away from the other view. Owing to the fact that this system example relies on trajectory preservation, the observer must be relatively far away (>2 m) to get a quality observation since the assumption is that the object is in the back focal plane implying the image is at infinity.
(54)
(55) Multiple-Depth Projection
(56) Another method for acquiring the background information that can be projected after postprocessing is discussed in further detail below. In some embodiments, a multiple-depth cloak can be achieved in some instances using a 3D image capture. For example, this may be done with stereoscopes or LIDAR. Once one obtains a depth map, then one can determine a length map or the distance the projected object needs to move across the screen as a function of observer angle. As shown in
(57) It might be of interest to consider what the ultimate limits for projection are given currently available technology. Consider a reasonable set of parameters. Suppose, the cloak is configured to work over a 30 degree angle with each degree yielding a unique view. This would correspond to a 10 pixel wide sub-pixel interlace (30 views). Assuming that there are approximately 100 pixels per inch (assuming a monitor approximately 40 inches wide) on a 4 k ultra television, this would correspond to a 10 lenses-per-inch array. Since the human visual acuity is approximately 0.0003 radians, this means that the observer would only need to be approximately 10 meters away from the cloak before the observer could not ascertain spatial features with sufficient clarity to distinguish the parsed projection of the background from the background assuming colors are properly matched. These are somewhat remarkable results considering the close proximity of the observer to the cloak and the relatively high number of viewing angles. With this many viewing angles, the changes to the views appear continuous.
(58) Discretized/Digital Cloaking
(59) Invisibility cloaking makes the cloaked object appear transparent, as if the light fields exited the cloaked space without the object present [3, 10]. It is a form of illusion, where the light bends around the cloaked space, but re-forms afterwards to appear as if it had never bent. This allows both the cloaked object and the cloaking device to not only be hidden, but appear transparent.
(60) With an ideal cloak (broadband, omnidirectional, 3D, phase-matching, etc.) a ray optics approximation may be made, where the full phase of the electromagnetic field of light is not necessarily matched. For imaging, whether by camera or by the human eye, the phase is typically not detectable, which is why ray tracing is usually sufficient for designing imaging devices. Ray optics cloaking can be considered a discretization of spectrum and phase for a given ray, since its phase (modulo 2n) will match for one or more discrete frequencies, or discrete phase values can be matched for a given frequency. Ray optics alone significantly reduces the complexities of cloaking such that isotropic, off-the-shelf materials can be used to build macroscopic cloaks for small angles [10].
(61)
(62) In the past, building an omnidirectional cloak has been elusive to demonstrate, even for ray optics. However, for practical usage, since detectors including the human eye have finite resolution, the appearance for omnidirectionality can be achieved by discretizing space and momentum (or angle). Given the finite resolution of detectors, such as the human eye, discretization can be unnoticeable. A rotationally symmetric example is shown in
(63) Implementing a discretized cloak or a digital cloak involves propagating the rays from input to output correctly. This can be done using a paraxial cloaking matrix (Equation (1) of Ref [10]), since the final ABCD matrix is still valid outside of the paraxial (small-angle) regime. This is also shown in
(64)
(65) Rotational symmetry about the center axis (z) is assumed and the ambient medium has refractive index n. Note that each ray has its own longitudinal distance L=(z.sub.fz.sub.i) that is dependent on its input and output planes for the cloak. To be direct, the real angle was used instead of the paraxial angle u(=tan ). Although
(66) For some implementations of the digital cloak, it may be useful to invert Equation (1):
(67)
(68) Equation (2) shows how to find the detector pixel that should be used for the display pixel positioned at y=y.sub.f, z=z.sub.f. For example, one may first select detector subpixels that collect rays of the same angle (.sub.i=.sub.f), color, and any other desired attributes as the output ray. Out of these candidate subpixels, one may select the one that has transverse (y) position closest to y.sub.i, and longitudinal (z) position closest to z.sub.i, where y.sub.i and z.sub.i satisfy Equation (2). Graphically, one may extend the dotted arrow in
(69) Demonstration of an Integral Cloak
(70) The following are examples of digital, discretized cloaking devices that detect and reproduce proper ray positions and angles. One example of a way to achieve this is to utilize Shack-Hartmann wavefront sensors, or fly's eye lens arrays. These allow the position and momentum of rays to be captured by using arrays of small lenses, which can spatially separate rays of different angles (See
(71) An example implementation of an integral cloaking using integral imaging techniques is shown in
(72)
(73)
(74)
(75)
(76) For 2D, a scanning camera may improve performance in some embodiments compared to a combination of lenslet and detector arrays (input surface of
(77) For the output display, a 20 cm (diagonal) LCD monitor (Apple iPad mini 4) was used. The output lenslet array is a 2D cylindrical lenslet array (20 lens-per-inch array from Micro Lens Technology). Both display monitor and lenslet array were commercially available. For a 3D integral cloak, a fly's eye lens array may be used to replace the cylindrical lenslet array. By slanting the cylindrical lenses, the 3 RGB subpixels were used to gain 3 times the horizontal angular resolution (in number of views), at the sacrifice of vertical resolution [15]. The output system generated 51.5 discrete views over 29 of viewing angles (field-of-view), horizontally. This 29 was the field-of-view of the lenslet array (FOV.sub.l), and limited the cone of angles for both the output and input of the cloaking system, since the input camera field-of-view was larger) (60. Each view may correspond to a discrete ray angle/momentum (one subpixel in
(78)
(79) The vertical magnification was matched for a particular observer distance and object depth combination, since this was a 2D cloak with cylindrical lenses. In this case, from the observation distances used in
(80)
(81) Finally, the digital integral cloak was characterized with additional quality metrics. Since the exemplary digital integral cloak was a 2D demonstration, the analysis was limited to the horizontal (x) and longitudinal (z) dimensions. The horizontal input angular resolution for the system was 0.031, which corresponds to the uncertainty in the input ray angles. (Recall the output angular resolution was 0.56.) To provide sufficient depth-of-field, the input camera was stopped-down to f-number=f/10. The resulting input aperture diameter was then 0.88 mm (effective lenslet diameter in
(82) Experimental Setup
(83) For the image capture (input) plane, a digital camera (Sony DSC-RX10) was used, mounted on a mechanical slider that scans horizontally at a fixed speed. Each camera frame represented a single lenslet and pixel (of the input surface in
(84) The output plane may be parallel to the input plane and the two planes may be separated by a distance L. The background objects included four sets of colored blocks with the dimensions shown in Table S1.
(85) TABLE-US-00001 TABLE S1 Object dimensions. Distance is from the input plane to the center of the objects, depth is the longitudinal (z) dimension, width is the horizontal (x) dimension, and height is the vertical (y) dimension. (All length units are in cm.) Objects Distance Depth Width Height Back wall (w/black cover) 98 Dark green blocks 87.6 3.8 3.8 33 Red blocks 70 5.7 2.5 35 Lime green blocks 47 3.8 3.8 33 Yellow blocks 40.6 3.8 3.8 29
(86) For horizontal (x) separations between the blocks, the farthest sides of the dark green blocks and the red blocks were 9 cm apart, and the farthest sides of the lime green blocks and the yellow blocks were 18 cm apart. The blocks, from the back side of the dark green blocks to the front side of the yellow blocks, spanned a total depth of about 51 cm. The distance from the front face of the yellow blocks to the camera sensor at the input plane was 39 cm.
(87) The Sony DSCRX10 had a 1 inch 20.2 MP Exmor sensor for low lighting, and a f/2.8 Carl Zeiss Vario-Sonnar T*24-200 mmlens with 8.3 optical zoom. It was mounted on a mechanical slider (from Varavon) that scans horizontally at a fixed speed, which can be adjusted. 18.5 cm of travel was used for this setup. Here, Ws=12 cm was the active screen width of the display (Apple iPad mini 4), and FOV.sub.l=29 degree was the field-of-view (FOV) of the output lenslet array (20 cylindrical lens-per-inch array from Micro Lens Technology).
(88)
(89) Ray Propagation Processing
(90) The propagation of rays over a finite distance is a characteristic of a cloak. After collecting the images from the input video scan, the video frames were processed using a code (written in C++) on a laptop (Lenovo Thinkpad W540).
(91) Steps to Propagate Rays
(92) Each input video frame was assigned an input ray position x.sub.i, that is given by the position of the input camera at the time the frame was captured. Each view represents an input ray angle .sub.i, which corresponded to a horizontal pixel position of the input video frame. Each valid input ray was then propagated using Equation (1) with (z.sub.fz.sub.i).fwdarw.L, while assuming ambient air (n=1). If the calculated output ray fell on a valid display pixel (e.g., an RGB subpixel), then the image pixel information for this input ray was mapped to the output display pixel. This then completed the ray propagation for the cloak.
(93) Inverting Propagation
(94) As set forth above, for digital cloaking, it may be useful to invert Equation (1) to equation (2). Equation (2) shows how to find the detector pixel that should be used for the display pixel positioned at y=y.sub.f, z=z.sub.f.
(95) Vertical Scaling
(96) Since the demonstration was a 2D cloak that used cylindrical lenses, the vertical pixel positions of the background objects were fixed. This implies that the vertical size, or magnification, relative to the screen size will be constant, even if the observer changes position only in the 2D x-z plane. However, at least the vertical magnification can be matched for a particular observer distance and object depth combination. For example, for the demonstrations shown in
(97) Output SurfaceOutput Display
(98) The output display monitor used was an Apple iPad mini 4. The display was an LED-backlit LCD monitor, with an active screen size of 20 cm (=7.9 inch) in the diagonal direction. The display pixel size was 20481536 resolution, at 326 pixels-per-inch (PPI). The display pixels were assumed to be square pixels, which was a fairly good assumption. This gives a width of 12.0 cm, and a height of 16.0 cm for the active screen size, in portrait mode. The 3 RGB (Red, Green, Blue) subpixels that made up a display pixel, were long in the vertical direction and short in the horizontal direction when the iPad mini was in portrait mode.
(99) Output SurfaceOutput Lenslet Array
(100) The 20 cylindrical lens-per-inch array from Micro Lens Technology used was a very thin (0.41 mm) sheet of plano-convex lenses. This was so manual adjustment of the separation distance between the lenslet array and the display screen was possible. The lenslet array was mounted on a transparent glass that was 2.3 mm thick, using the adhesive layer on the flat side of the lenslet array. The distance between the lenslet array and the screen was adjusted so that the LCD screen pixels lied at the back focal plane of the lenslet array. The field-of-view of the output lenslet array (FOV.sub.l) given by the manufacturer was 29 degree.
(101) Output SurfaceSlanted RGB Subpixel Imaging
(102) The cylindrical lenses of the output lenslet array were slanted at an angle of 3 pixels vertically down, 1 pixel horizontally across. This then uses the 3 RGB subpixels of the display to gain 3 times the horizontal angular resolution (in number of views), at the sacrifice of vertical resolution. The particular combination of slanted lenslet array and display monitor generated 51.5 discrete views over 29 degree of viewing angles (field-of-view). With this slanted setup, the total number of views (DimViewX.sub.s), for a given display monitor and lenslet array combination, is given by:
(103)
PPI (pixels-per-inch) is the pixel density of the display monitor, and LPI (lens-per-inch) is the lens density of the cylindrical lenslet array used. Equation (3) assumes that the display pixels are uniform and square in dimension.
(104) Horizontal (x) Demonstration
(105) The details for the horizontal demonstration shown in
(106) Longitudinal (z) Demonstration
(107)
(108)
where W.sub.s is the horizontal width of the output display screen of the cloak, and z.sub.obs is the longitudinal distance of the observer from the output screen (the output plane, which is set to z=0). The resulting FOVx.sub.observer values were then 2.53, 2.93, 3.38, 4.59, for
(109) The images shown in
(110) Characterization Metrics
(111) Details of some of the calculations made in the main text are presented below, for the metrics that were used to characterize the digital integral cloak. Since the exemplary cloak was a 2D demonstration, the analysis is limited to the horizontal (x) and longitudinal (z) dimensions.
(112) A. Output Surface Metrics
(113) A.1. Output Spatial Resolution
(114) The limiting spatial resolution of the cloaking system was given by the output lenslet array. The spatial resolution then can simply be calculated by inverting the lens per length value of the lenslet array. So for the 20 lens-per-inch (LPI) lenslet array used, the spatial resolution is given by 1.27 mm=1/(20 lens/inch). This is the width of a single lens of the output lenslet array. However, this is true only if the cylindrical lenslet arrays were not slanted at an angle. Since RGB subpixel interlacing was used, the lenslet array was slanted at an angle of 1 pixel across (horizontally) to 3 pixels down (vertically). The corrected horizontal spatial resolution (dX.sub.s in inches) is then given by
(115)
(116) The limiting spatial resolution of the exemplary cloak, calculated from Equation (5), is then 0.0527 inch=1.34 mm.
(117) A.2. Output Display Pixel Spacing
(118) The output display monitor (iPad mini 4) had a pixel density of 326 pixels-per-inch (PPI). So the pixel spacing resolution (horizontal or vertical) of the output display is 0.078 mm. Since RGB subpixel interlacing was used, the 3 RGB subpixels were actually used, giving a finer spatial resolution of 0.078/3 mm=0.026 mm per RGB subpixel.
(119) This small pixel resolution was used in calculating the propagated ray positions (Equation (1)) more precisely than the spatial resolution of the lenslet array (Equation (5)), for the exemplary cloaking demonstration. However, using the spatial resolution in Equation (5) also seemed to provide similar images, likely due to the relatively small spacing of both values compared to the observation distances.
(120) A.3. Output Angular Resolution
(121) The limiting angular resolution of our cloaking system was given by the output surface (display+lenslet array). The angular resolution (d.sub.0) is simply:
(122)
(123) FOV.sub.l is the field-of-view (FOV) of the lenslet array, and DimViewX.sub.s is the total number of views from Equation (3). For the exemplary digital integral cloak, d.sub.0=0.56.
(124) B. Input Surface Metrics
(125) B.1. Ideal Input System
(126) First, the quality of input rays captured rely on the ability to separate position and angle (momentum) precisely, with no cross-talk. This is done by focusing the input camera to infinity (see
(127) B.2. Input Spatial Resolution
(128) For the input scan, 60 frames per second were captured, while scanning at 6.35 mm/s. This high frame rate and slow scan gave a horizontal spatial resolution (dX.sub.c) of 0.106 mm for the spacing between input camera positions:
(129)
(130) Here, .sub.scan-s is the speed of the scan in units of (length/second), and FPS.sub.c is the frame-per-second frame rate of the input camera.
(131) B.3. Input Angular Resolution
(132) Focusing to infinity maps the same ray angles to the same detector pixels of the input camera. Hence the input ray angles are then limited by both the aberrations of the camera and the detector pixels. With a highly aberration-corrected Carl Zeiss lens, aberrations were assumed to be minimal. Then the horizontal input angular resolution (d.sub.i) is given by
(133)
(134) where FOV.sub.c is the horizontal field-of-view of the input camera, and DimX.sub.f is the total number of horizontal pixels for the input video frame. d.sub.i corresponds to the uncertainty in the input ray angles. For the exemplary cloak, d.sub.i60 degree/1920=0.031 degree.
(135) B.4. Input Aperture
(136) Next, to ensure that each transverse object position was mapped to separate detector pixels the aperture of the input camera may be stopped down. Doing so decreases the aperture size, reducing the transverse (x, y) spatial extent of the rays that are captured by the input camera. Otherwise, blurring of the input images occur, i.e., a reduced depth-of-field results. It is well known in optics and photography that smaller f-numbers (=focal length/aperture size) or reduced aperture size, improves the depth-of-field, which is how clear (in-focus) the objects appear. Ideally, the aperture size should be infinitely small, but then the amount of light entering is infinitely small, too. So this was balanced by using an f-number (f/#)=f/10.
(137) With the input f/#, the diameter of the input aperture stop (CA.sub.i) used may be calculated:
(138)
(139) Here, f is the focal length used. The specifications for the Sony DSC-RX10 input camera stated the focal length range to be f=8.8-73.3 mm. Since the camera was zoomed all the way out, f=8.8 mm. This then gives an input aperture size CA.sub.i=0.88 mm. CA.sub.i gives roughly the range of transverse spatial positions of the objects that are captured for each detector pixel of the input camera. This is particularly true when the lens is focused to objects at infinity, since only a single collimated ray bundle will enter the lens and be focused onto a single detector pixel (ignoring diffraction and aberrations). So CA.sub.i in this way is proportional to the blurring of objects in the input images.
(140) The number to compare to CA.sub.i would be the aperture set by the output surface, which is given by the spacing of a single lens in the output lenslet array (1.34 mm), or the output spatial resolution (Equation (5)). Again, the output lenslet array can be seen to be the limiting element for spatial resolution and blurring of objects in the cloak. It is worth noting that CA.sub.i is different than the positional uncertainty of the input camera; the latter is the uncertainty for each input ray position, and is given by the input spatial resolution (Equation (7)) of the camera scan determined earlier (0.106 mm).
(141) B.5. Input Depth of Field
(142) The depth-of-field of the exemplary input system is discussed briefly.
(143) C. Other System Metrics
(144) C.1. Sensitivity to Misalignment
(145) Misalignment in hardware setup and imprecise propagation calculations can reduce the cloak quality. These can physically be manifested in skewed angles, shifts in object positions, or mismatching speeds of object movements as seen on the cloak, when compared to the actual background without the cloak. The demonstrated system seemed to be most sensitive at the output surface. This is mainly due to the high pixel density (326 pixels-per-inch) of the output display. So the horizontal spatial sensitivity (dx.sub.mis) of an unslanted lenslet array to be misaligned on the display is such that a 0.078 mm change in position will shift the view seen by an observer. For slanted lenslet arrays, dx.sub.mis is 3 times more sensitive (0.026 mm).
(146) Ideally, the angle of the lenslet array should align exactly along the display pixel line. Even 1 pixel change across the total height of the display screen will begin to cause deviations from the ideal image. So the angular sensitivity (d.sub.mis) of the alignment of the lenslet array with respect to the display screen pixels is
(147)
where PPI is the pixels-per-inch linear pixel density, and Hs.sub.in is the active display screen height in inches. Equation (10) is for slanted lenslet array mounting on the display. For unslanted lenslet mounting, the (3/10) factor in arctan should be removed. For the exemplary demonstrated digital integral cloak with slanted lenslets, d.sub.mis=(8.810.sup.3). Increased resolution of displays used may require increasingly careful mounting of the lenslet array on the display screen, in both relative positioning and angular alignment.
(148) C.2. Other Factors
(149) Other causes of reduced image quality for the cloak include non-uniform bending of the lenslet array plane toward or away from the display screen, and aberrations of the lenslet arrays themselves. Since easily accessible lenslet arrays are spherical or cylindrical in curvature, even spherical aberrations are not corrected. These aberrations effectively increase the depth and transverse size of the focal point of the lenslet arrays. The result is that the display pixels will not be in focus and multiple display pixels will mix, causing blurring of the output image. Even simple lens optimizations of the lenslet array surfaces should improve the cloaking quality.
(150) Real Time Digital Cloaking
(151) Real-time digital cloaking can be achieved using detector and display pixels combined with lenslet arrays (
(152)
(153) Display pixels currently can be obtained commercially with high density in pixels per length/area. Lenslet arrays that are relatively dense in number of lenses per length/area can also be obtained commercially. These components may not be too expensive as they are used for consumer products. However, what is perhaps expensive to build and obtain may be the detector array, or an array of small cameras, to capture input rays. To build a practical real-time cloak, commercially available cameras may be utilized. The one or more cameras may be placed sparsely, and then the camera images may be interpolated to estimate the input rays that pass between the cameras (which are not captured).
(154) Connecting cameras and processing them in real-time may also be tricky, for large number of cameras. One scalable method to connect the cameras is to use internet protocol (IP) cameras or detectors combined with switches and/or routers. The input ray collection system is then akin to a network of cameras on the internet. Many connections can be made, including very large number of cameras/detectors, just like there are a large number of computers and devices connected on the internet. Fast bandwidth can be obtained with commercially available routers and switches. This may not be as large of a bottleneck as one might assume for connecting a large number of devices simultaneously in real-time. This is because the image from one camera may be pulled at a time, the image may be processed for output, and then the process could proceed to the next camera and so forth, rather than pulling images from all cameras/detectors at the same time. More camera images may be pulled simultaneously, up to the bandwidth of the connection. When the bandwidth limit of the connection is reached, parallel sets of routers and/or switches, corresponding to separate sets of cameras, may be used to increase the effective bandwidth of the connection. Regardless, the input collection time for all cameras/detectors, and the processing time will cause some delay to updating the output of the cloaking device. However, since a human observer or a machine has finite temporal resolution, the cloaking device can use the methods suggested to provide cloaking that appears real-time for the observer, as long as the delays are not noticeable.
DISCUSSION
(155) The digital cloak demonstration discussed above for
(156) In some implementations, the phase of the light fields can be matched by including properly engineered materials for a fixed-shape cloak, or spatial light modulator arrays for a cloak with dynamic shapes. If each subpixel is assumed to correspond to a single ray position, angle, frequency, an input subpixel may be traced to its output subpixel (Equation (1)). To good approximation, each pair is then a unidirectional propagation from input subpixel to output subpixel (dotted lines in
(157) Other methods for discretized or digital cloaking are also possible. Optical fibers or other optics, with good anti-reflection coatings, can be used to collect and transmit analog measurement values. Some advantages of this include cloaking in a passive (rather than active) manner, and that its spectrum can be continuously broadband. Additionally, a digital cloak without lenses is possible by using curved surfaces for detection and emission of rays. This is similar to one of the original methods proposed by Lippman for integral photography [16]. An example surface of such a cloak is shown in
(158) In conclusion, to approximate an ideal cloak for practical observation, discretized cloaking is proposed. In particular, a 2D digital cloak was demonstrated, called an integral cloak, that uses commercially available digital technologiesA camera to capture the input rays, and a monitor+cylindrical lenslet array for the output rays. The principles for generating a 3D integral cloak follow easily. Although the demonstration was for ray optics cloaking, other designs are suggested, including methods to match the phase of the light fields. Digital cloaking has good potential for wide implementation as a wearable cloak, since the digital technology required continue to improve commercially.
(159) Those of skill in the art will recognize that the systems and methods described above can incorporate one or more computing devices, whether stand-alone computers, networked devices, or devices that are fully or partially integrated into the other devices and components described above. Computing devices may include a processor, a memory, and a bus. The memory may include a tangible, computer-readable memory on which code is stored. The processor may execute code stored in the memory by communication via the bus to cause the computing device to perform actions. The computing device may include an input/output (I/O) interface for communication with other components. The computing device may be any device that can process data and execute code that is a set of instructions to perform actions.
(160) It will be appreciated that variants of the above-disclosed and other features and functions, or alternatives thereof, may be combined into many other different systems or applications. Various presently unforeseen or unanticipated alternatives, modifications, variations, or improvements therein may be subsequently made by those skilled in the art which are also intended to be compassed by the following claims. The claims can encompass embodiments in hardware, software, or a combination thereof.
REFERENCES, EACH OF WHICH IS INCORPORATED HEREIN BY REFERENCE IN THEIR ENTIRETY
(161) 1. G. Gbur, Invisibility physics: Past, present, and future, Progress in Optics 58, 65-114 (2013). 2. J. B. Pendry, D. Schurig, and D. R. Smith, Controlling electromagnetic fields, Science 312, 1780-1782 (2006). 3. U. Leonhardt, Optical conformal mapping, Science 312, 1777-1780 (2006). 4. M. McCall, Transformation optics and cloaking, Contemporary Physics 54, 273-286 (2013). 5. R. Fleury, F. Monticone, and A. Alu, Invisibility and cloaking: Origins, present, and future perspectives, Physical Review Applied 4, 037001 (2015). 6. J. S. Choi and J. C. Howell, Paraxial full-field cloaking, Optics Express 23, 15857-15862 (2015). 7. J. S. Li and J. B. Pendry, Hiding under the carpet: A new strategy for cloaking, Physical Review Letters 101, 203901 (2008). 8. N. Landy and D. R. Smith, A full-parameter unidirectional metamaterial cloak for microwaves, Nature Materials 12, 25-28 (2013). 9. J. C. Howell, J. B. Howell, and J. S. Choi, Amplitude-only, passive, broadband, optical spatial cloaking of very large objects, Applied Optics 53, 1958-1963 (2014). 10. J. S. Choi and J. C. Howell, Paraxial ray optics cloaking, Optics Express 22, 29465-29478 (2014). 11. R. Schittny, M. Kadic, T. Bueckmann, and M. Wegener, Invisibility cloaking in a diffusive light scattering medium, Science 345, 427-429 (2014). 12. M. Bass, J. M. Enoch, and V. Lakshminarayanan, Vision and vision optics, vol. 3 of Handbook of optics (McGraw-Hill, New York, 2010), 3rd ed. 13. F. G. Vasquez, G. W. Milton, and D. Onofrei, Active exterior cloaking for the 2d laplace and helmholtz equations, Physical Review Letters 103, 073901 (2009). 14. G. Lippmann, Epreuves reversibles. photographies integrales, C. R. Acad. Sci. 146, 446-451 (1908). 15. J. Geng, Three-dimensional display technologies, Advances in Optics and Photonics 5, 456-535 (2013). 16. Integral photography, Scientific American 105, 164 (1911).