RECOMPRESSIVE SENSING, RESPARSIFIED SAMPLING, AND LIGHTSPACETIMELAPSE: MEANS, APPARATUS, AND METHODS FOR SPATIOTEMPORAL AND SPATIOTONAL TIMELAPSE AND INFINITELY LONG MEDIA OR MULTIMEDIA RECORDINGS IN FINITE MEMORY
20190158777 ยท 2019-05-23
Inventors
Cpc classification
H03M7/30
ELECTRICITY
G11B27/031
PHYSICS
International classification
Abstract
A recompressed-sensing recording means, apparatus, device, or system captures one or more recordings, of possibly unknown or unbounded duration, into a finite memory, by resparsifying previously recorded sensor data in order to make room to store new incoming sensor data. In some embodiments this resparsification is recursive, resulting in a fraccular (fractally circular) buffer. In some embodiments a LightSpaceTimeLapse image capture means, apparatus, or system captures the passage or stoppage of time, by way of analyzing a scene or subject matter that is subject to changes in lighting or changes in the subject matter, or both. In some embodiments a sparse or reduced resolution test image is captured periodically, and a LightSpaceTime model is constructed to estimate changes in LightSpace or SpaceTime or both. Successive frames feed into a LightSpaceTime comparator which triggers full resolution capture into a finite memory at appropriate intervals. As the finite memory capacity approaches full capacity, the image repository is resparsified to make room for more new images. This resparsification is done by a decision process based on LightSpace and SpaceTime analysis. In some embodiments an intermediate LightSpaceTime format is captured and rendered as a background task resulting in an optimization that varies over time, depending on what is considered important in the LightSpaceTime continuum. All exposure and time values are preserved allowing an interpolated reconstruction at constant framerates or constant noveltyrates, as may be desired for artistic or epistemological purposes or for forensically accurate and irrefutable evidence.
Claims
1. A multimedia recorder, said recorder including a sensor, means for capturing data from said sensor, a processor responsive to an input from said recorder, and memory for storage of said data, said recorder including a resparsifier, said resparsifier pruning older recordings in said data, to higher pruning levels, in response to a fullness of said memory.
2. The recorder of claim 1, where said pruning comprises temporal downsampling of said recordings.
3. The recorder of claim 2, where said temporal downsampling comprises the deletion of the oldest even numbered audiovisual or still image frame or sample.
4. The recorder of claim 2, where said temporal downsampling comprises the deletion of an oldest odd numbered frame or sample.
5. The recorder of claim 1, where said pruning comprises the recompression of the oldest frame or sample, to make room for each arriving new frame or sample.
6. The recorder of claim 1, where said pruning comprises the downsampling of the oldest frame or sample, to make room for each arriving new frame or sample.
7. The recorder of claim 1, where said pruning comprises the downgrading of older frames or samples, to make room for each arriving new frame or sample.
8. The recorder of claim 1, where said pruning comprises the downgrading of every-other frame or sample, to make room for each arriving new frame or sample.
9. The recorder of claim 1, where said pruning comprises a spatiotemporal downsampling of older frames to make room for new arriving frames.
10. The recorder of claim 1, where said pruning comprises a spatiotonal downsampling of older frames or samples to make room for new arriving frames or samples.
11. The recorder of claim 1, where said pruning comprises a tonaltemporal downsampling of older frames to make room for new arriving frames.
12. The recorder of claim 1, where said pruning comprises a spatiotonaltemporal downsampling of older frames to make room for new arriving frames.
13. The recorder of claim 1, where said recorder includes a comparator comparing pairs of incoming images, said processor capturing from said recorder, sparse images, said comparator comparing said sparse images, said processor responsive to an output of said comparator, said processor capturing a full image when said comparator produces a difference beyond a similarity threshold.
14. The recorder of claim 13, where said pruning comprises a novelty downsampling of older frames to make room for new arriving frames.
15. The recorder of claim 13, where said pruning comprises a dissimilarity downsampling of older frames to make room for new arriving frames.
16. A Light Space Time Lapse camera service, said service including an image collector for collecting images from a user or subscriber camera, a processor responsive to an input from said camera, and memory for storage of said data, said service including a resparsifier, said resparsifier pruning older recordings in said data, in response to a fullness of said memory.
17. The service of claim 16, where said memory is not necessarily physically limited memory but rather virtually limited based on financial resources of the user.
18. A Light Space Time Lapse camera system said system including a means for collecting images from a camera, and means for storing the images in a memory, said system including means for resparsification said means for resparsification pruning older images, in response to a fullness of said memory.
19. A multimedia recording device for capturing for a possibly unknown or unbounded duration, into a bounded or finite memory, said device including a sensor, means for capturing data from said sensor, a processor responsive to an input from said sensor, and memory elements for storage of said data, said device including a resparsifier responsive to a fullness of said memory.
20. The recording device of claim 19, said resparsifier downgrading older recordings as new recordings are made from said sensor.
21. The recording device of claim 19, said resparsifier downsampling older recordings as new recordings are made from said sensor.
22. The recording device of claim 19, said resparsifier downcompressing older recordings as new recordings are made from said sensor.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0064] The invention will now be described in more detail, by way of examples which in no way are meant to limit the scope of the invention, but, rather, these examples will serve to illustrate the invention with reference to the accompanying drawings, in which:
[0065]
[0066]
[0067]
[0068]
[0069]
[0070]
[0071]
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0072] While the invention shall now be described with reference to the preferred embodiments shown in the drawings, it should be understood that the intention is not to limit the invention only to the particular embodiments shown but rather to cover all alterations, modifications and equivalent arrangements possible within the scope of appended claims.
[0073] In various aspects of the present invention, references to microphone can mean any device or collection of devices capable of determining pressure, or changes in pressure, or flow, or changes in flow, in any medium, be it solid, liquid, or gas.
[0074] Likewise the term geophone describes any of a variety of pressure transducers, pressure sensors, velocity sensors, or flow sensors that convert changes in pressure or velocity or movement or compression and rarefaction in solid matter to electrical signals. Geophones may include differential pressure sensors, as well as absolute pressure sensors, strain gauges, flex sensors on solid surfaces like tabletops, and the like. Thus a geophone may have a single listening port or dual ports, one on each side of a glass or ceramic plate, stainless steel diaphragm, or the like, or may also include pressure sensors that respond only to discrete changes in pressure, such as a pressure switch which may be regarded as a 1-bit geophone. Moreover, the term geophone can also describe devices that only respond to changes in pressure or pressure difference, i.e. to devices that cannot convey a static pressure or static pressure differences. More particularly, the term geophone is used to describe pressure sensors that sense pressure or pressure changes in any frequency range whether or not the frequency range is within the range of human hearing, or subsonic (including all the way down to zero cycles per second) or ultrasonic.
[0075] Moreover, the term geophone is used to describe any kind of contact microphone or similar transducer that senses or can sense vibrations or pressure or pressure changes in solid matter. Thus the term geophone describes contact microphones that work in audible frequency ranges as well as other pressure sensors that work in any frequency range, not just audible frequencies. A geophone can sense sound vibrations in a tabletop, scratching, pressing downward pressure, weight on the table, i.e. DC (Direct Current) offset, as well as small-signal vibrations, i.e. AC (Alternating Current) signals.
[0076]
[0077] Spime 110 is defined by spatial axes Space X, Space Y, and Time T which set forth as a stack of pictures that have spatial dimensions X and Y, acquired along a temporal dimension T. Here we consider a camera sensor array, but equally valid is to consider RaDAR (Radio Direction And Ranging), LiDAR (Light Direction And Ranging), SoNAR, ToF (Time of Flight), or other multidimensional image capture devices that capture spatiotemporal fields of sensory information. A satisfactory sensor is a CCD (Charge Coupled Device) array or the like, which can be read sparsely or at reduced resolution, at low power consumption.
[0078] Images in spime 110 are denoted as Testframe1, Testframe2, . . . through to Testframe6, although in practice the frame count goes much higher. These testframes denoted as Testframes 120 are each a snapshot in space, at a particular instant in time. Pairs of testframes are analyzed in terms of features, by way of a feature extractor in a processor, responsive to an input from the camera. The processed images produce a signal vector or feature vector, as per, for example, Lightspace (Intelligent Image Processing textbook by Steve Mann, published by Wiley, 2001), or VideoOrbits, or any other suitable feature system. This results in signals 131 and 132 here by example from Testframe1 and Testframe2. These signals are compared and a comparison signal 133 indicates how similar the testframes 120 are. When there is determined to be enough novelty, by way of a novelty threshold on comparator 130, the processor captures a full image set at full resolution, and these full exposures are synthesized into Keepframes 140. Keepframes are full-resolution frames generated when there is enough new information in the scene to warrant this. These are denoted in spime 150.
[0079] A common problem in timelapse photography is light flicker. Thus a lightspace model is set forth and in the Testframes may sense for example sun and cloud cover, etc., and try and capture images when the sun is shining, so that they all look the same more or less. Thus on a day when there is sun and cloud moving cover, we try to grab pictures at the instant the sun comes through the clouds, and mark these as preferred images evicted last. So as the eviction happens (when the memory is near full) we start to prune the flickery outliers and keep more of the steady images that are more similar to each other in lightspace but more different from each other in scene novelty.
[0080] This happens through Machine Learning and AI (Artificial Intelligence), whereby the processor looks to favour capture of images that are steady but different (novel): [0081] Steady: we want images that are captured at an instant in which the signal is high and the noise is low; and [0082] Novel: we want a set of images in which each new image conveys a reasonable novelty, i.e. we don't want thousands of pictures of the same empty space with no change between them.
[0083] We can think of this as a signal-to-noise ratio, i.e. change due to flicker is what we don't want, and change due to actual scene subject matter movement is what we do want.
[0084] Note that the change in shadows is often signal not noise, i.e. the smooth graceful movement of the shadows in a scene is quite nice artistically, once the flicker of clouds is filtered out.
[0085] Cloud movement itself can also be nice if in a smooth and steady fashion, while revealing the right amount of novelty.
[0086] As can be seen, there is a process for measuring novelty. Each image has a header, e.g. in the JPEG header, we can store the novelty and various sensor and processor parameters and information like timestamp, and the like, that helps us later on.
[0087]
[0088] The matrix is not necessarily required to be stored in a memory to implement the invention, but serves as a way to understand and precisely specify enabling aspects of the invention.
[0089] Rows of the matrix indicate points in time, i.e. each row indicates a particular point in time. The points in time need not necessarily be equally spaced apart in time. In some embodiments the time samples are based on innovation in the subject matter being recorded. Audio, visual, or other scene innovation, in some embodiments, is measured. For images or video or pictures, this is done according to a distance of an orbit of a projective group of coordinate transformations, such as, for example, variation along an orbit of algebraic projective geometry such as might happen when the scene changes enough, as outlined in the textbook Intelligent Image Processing, author S. Mann, publisher Wiley, 2001. In some embodiments a full fidelity sample or image frame is grabbed at time t.sub.1, and stored, and then new incoming images are grabbed at low fidelity to save battery power. Each of these incoming image frames is compared with the one grabbed at time t.sub.1 and when there is sufficient difference or sufficient novelty or sufficient innovation, there is another full fidelity image frame captured at what is declared as time t.sub.2. The process continues, filling up each memory element with sufficiently different information so as to capture the activity in a space or scene or subject matter in front of the camera or cameras to which the apparatus is applied. In some embodiments there are multiple synchronized cameras. In some embodiments there are also multiple synchronized light sources, and the novelty or innovation of the image subject matter is sensed in lightspace as well as imagespace, where lightspace is defined as per the above-mentioned Intelligent Image Processing textbook.
[0090] The top row, in
[0091] Here in
[0092] Columns of the TimeEl Matrix indicate the relationship between memory element usage and time. Rows and columns both represent the same units of time, i.e. the reciprocal of the frame rate. In this scenario there is sufficient memory to capture 8 frames, so during a first (re)sparsification regime, s.sub.0, each new frame comes in, as frame number f.sub.n, where 0<n8, i.e. frames f.sub.1 through f.sub.8 and each frame is recorded into memory elements e.sub.n for 0<n8. During the first 8 frame captures, the matrix is lower-triangular. This situation is denoted as (re)sparsification regime S.sub.0. (Re)sparsification regimes 210 comprise four regimes that are shown in this
[0093] In the field of surveillance video recordings, the concept of a circular buffer is commonly used in which the last L frames are kept, for some value of a memory element array length, L, such as in our case L=8. In this case, each time a new frame comes in we would delete (evict) the oldest frame to make room for the new frame. In this way, we have retroactive recording where we can stop recording and always have the most recent 8 frames. This works well for police encounters where maybe there is a shooting and then we stop the recording and retrieve the pictures of the shooting.
[0094] In shootings and mass murders, or jail, prison, riots, etc., we have a world in which there are years of boredom punctuated by seconds of terror.
[0095] However, in more general situations, from ordinary life, as well as in artistic applications, there might not be one specific incident of terror that punctuates an otherwise irrelevant and boring timeline. It might be, for example, that the entire timeline and context is of interest, not just a recent event.
[0096] In this case, what is shown in
[0101] Let us consider the latter, i.e. resparsifier 211 deletes some of the oldest images, as per rembrance evictor denoted as revictor, 201, to make room for new images. Rather than delete the oldest images only, it reduces the frame rate, so at time t.sub.0, the second oldest memory element is cleared. Then at time t.sub.10, the fourth oldest element is cleared. Note that at time t.sub.10, the second oldest memory element remains cleared. More generally, once an image is cleared, it is lost, and therefore its absence continues to be manifest itself. Thus at time t.sub.10, we see that both the 2nd oldest and the 4th oldest elements are depicted as cleared. We have emphasized this with the hand-drawn X in the timeline, but, more generally, this is not necessary as we simply represent this clearing by not showing a black dot in the matrix. Blank space in the matrix indicates emptiness, i.e. a reduction (or complete lack) of image content by way of by resparsifier 211.
[0102] Resparsification regimes 210 each have their frame-rates: [0103] Resparsification regime s.sub.0 is none, i.e. elements e.sub.i correspond directly to frames f.sub.m, so i=m; [0104] Resparsification regime s.sub.1 is eviction of even numbered frames, so that ultimately (by time t.sub.15) only odd-numbered frames are keptj; [0105] Resparsification regime s.sub.2 is eviction of even numbered of the even numbered frames, so that only odd-odd frames are kept, i.e. untimately (by time t.sub.22), only every 4th frame is kept; [0106] Resparsification regime s.sub.3 is eviction of even numbered of the even numbered of the even numbered frames, so that only odd-odd-odd frames are kept, i.e. untimately, only every 8th frame is kept.
[0107] Rembrance regimes, 200, each define a specific slope in the matrix: [0108] In rembrance regime r.sub.1 the ratio of the rate of frame capture and growth of the matrix define a slope of 1, meaning that each time a new frame comes in, the matrix grows by one unit in width. In this regime revictor 201 schedules resparsifier 211 to delete even frames and keep odd frames; [0109] In rembrance regime r.sub.2 the slope is 2. In this regime revictor preserves one in four frames; [0110] In regime r.sub.3 the slope is 4, and the revictor preseves one in eight frames.
Note that in matrix indexing, the first index (the X-axis) runs down the page, and the second index, (the Y-axis) runs across the page, so slope zero runs down the page and an infinite slope runs across the page.
[0111] Thus specifying a TimeEl Matrix specifies an algorithm for operating a timelapse camera system. The TimeEl Matrix is thus isomorphic to a very precisely defined algorithm that can be implemented in hardware or software or firmware.
[0112] Thus if we can precisely define a TimeEl Matrix, we have precisely specified an algorithm to allocate incoming frames of video to memory elements, such as to facilitate infinite recording into finite memory.
[0113] A TimeEl Matrix a is constructed as follows: Consider the example of a memory capacity of 4 frames. Initially these four frames fill up, e.g. into the first four rows of a, as follows (this code will run in Octave or Matlab, or the like):
a(1, 1)=1;
a(2, 1:2)=1;
a(3, 1:3)=1;
a(4, 1:4)=1;
[0114] Now that the memory is full, we drop to half the frame rate, while evicting (deleting to make space) every second image, after the first (oldest) image which we prefer to keep:
a(5, 1:5)=1; a(5,2)=0;
a(6, 1:6)=1; a(6,2:2:4)=0;
a(7, 1:7)=1; a(7,2:2:6)=0;
and the last line of code may be alternatively written as:
a(7, 1:2:4*2)=1;
which, in either case, deletes every second image, while thus recording at a frame rate half what was recorded for the first four images. Thus far we have defined the first seven rows of the TimeEl Matrix. We continue to the next three rows, by capturing at a still lower frame rate of one frame in four (quarter frame rate), and deleting the evens of the evens (i.e. keeping only evens, and deleting even evens, thus deleting every 4th image) as follows:
m a(8, 1:2:5*2)=1; a(8, 3)=0;
a(9, 1:2:6*2)=1; a(9, 3:4:4*2)=0;
a(10,1:2:7*2)=1; a(10, 3:4:6*2)=0;
a(10,1:4:4*4)=1; % same as the line above
and then dropping to quarter frame rate capture and making room by deleting every eight image:
a(11,[1:4:5*4])=1; a(11,5)=0;
a(12,[1:4:6*4])=1; a(12,5:8:4*4)=0;
a(13,[1:4:7*4])=1; a(13,5:8:6*4)=0;
a(13,1:8:4*8)=1; % same as the line above and then dropping to eight frame rate of capture and deleting every sixteenth image:
[0115] a(14, 1:8:5*8)=1; a(14, 9)=0;
[0116] a(15, 1:8:6*8)=1; a(15, 9:16:4*8)=0;
[0117] a(16, 1:8:7*8)=1; a(16, 9:16:6*8)=0;
[0118] and then dropping to a reciprocal frame rate, r=16, which corresponds to one sixteenth the original frame rate, and deleting every 2r=32nd image:
m=17;
r=16; % reciprocal frame rate (skip)
a(m,1: r:5*r)=1; a(m, r+1)=0; m=m+1;
a(m,1: r:6*r)=1; a(m, r+1:2*r:4*r)=0; m=m+1;
a(m,1: r:7*r)=1; a(m, r+1:2*r:6*r)=0;
and so on, continuing . . . .
[0119] Let b be the base 2 logarithm of the reciprocal frame rate r, so that we can let b go from 0 (i.e. r=1) onwards and upwards as the recording time increases all the way up to B:
m a(1, 1)=1;
a(2, 1:2)=1;
a(3, 1:3)=1;
a(4, 1:4)=1;
m=5;
n for b=0:B % log base 2 of the reciprocal frame rate, r
[0120] r=2 . . . b;
[0121] a(m, 1:r:5*r)=1; a(m, r+1)=0; m=m+1;
[0122] a(m, 1:r:6*r)=1; a(m, r+1:2*r:4*r)=0; m=m+1;
[0123] a(m, 1:r:7*r)=1; a(m, r+1:2*r:6*r)=0; m=m+1;
[0124] end % for b
[0125]
where the fourth row, seventh row, and tenth row correspond to capture at r=1, r=2, and r=4, respectively.
[0126]
[0133] Here the frame rate does not decay as it did in the TimeEl Matrices of
[0134] Memory fades away logarithmically, rather than linearly, so that there is always a semimonotonically decaying frame-rate as we go backwards in time. In other words, as we go backwards in time, the frame rate either stays the same or decreases. Thus yesterday's memory is about half as good as today's memory, and the memory from the day before is about a quarter as good as today's memory, and so on.
[0135] Timel Matrix 410 is shown as a 40 by 40 matrix which thus corresponds to 40 time units and 40 memory element usages. The first four rows of Timel Matrix 410, corresponding to time t.sub.1 trough to time t.sub.4 prescribe perfect memory at full frame rate, after which memory is (re)sparsified by resparsifier 411, according to a logarithmic memory fade, very similar to the way that human memory works, in the sense that as memories get older, they fade more.
TABLE-US-00001 M=40; N=40; % size of TIMEL MATRIX 410 a=zeros(M,N); % initialize TIMEL MATRIX 410 for m=1:M; a(m,[1:m])=1; end % initialize lower triangular portion to ones % the following 36 lines of code are the action of SPARSIFIER 411 a(5 :M, 2)=0; % *** zero out the first even column from 5th row downwards a(6 :M, 4)=0; % zero out the next even column from 6th row downwards a(7 :M, 3)=0; % *** need to keep at least 2 recent frames, so drop back to 3 a(8 :M, 6)=0; a(9 :M, 7)=0; a(10:M, 8)=0; a(11:M, 5)=0; % *** next would result in non-semimonotonicity, so drop 5 a(12:M,10)=0; a(13:M,11)=0; % delete newer room when possible, i.e. preserve oldest a(14:M,12)=0; a(15:M,13)=0; a(16:M,14)=0; a(17:M,15)=0; a(18:M,16)=0; a(19:M, 9)=0; % *** next would result in non-semimonotonicity, so drop 9 a(20:M,18)=0; a(21:M,19)=0; a(22:M,20)=0; a(23:M,21)=0; a(24:M,22)=0; a(25:M,23)=0; a(26:M,24)=0; a(27:M,25)=0; a(28:M,26)=0; a(29:M,27)=0; a(30:M,28)=0; a(31:M,29)=0; a(32:M,30)=0; a(33:M,31)=0; a(34:M,32)=0; a(35:M,17)=0; % *** next would result in non-semimonotonicity, so drop 17 a(36:M,34)=0; a(37:M,35)=0; a(38:M,36)=0; a(39:M,37)=0; a(40:M,38)=0;
[0136] SPARSIFIER 411 is manifest in the last 36 lines of code above, where we can see that most of this is clearing down column m2 from m down to M (i.e. down to the bottom of the matrix). Only five of these 36 lines of code prescribe otherwise. These five lines are marked in the comments with ***. The first of these anomalies is to zero out the first even column from the 5th row downwards. The second one is to keep at least two frames in recent memory, i.e. that L.sub.r=2. Beyond that, the anomalies are to preserve the semi-monotonicity requirement.
[0137] After the first four rows of perfect memory, the next two rows after time t.sub.4 prescribe odd memory (forgetting even frames) in non-recent memory. This ends at time t=t.sub.6, with perfect recent memory and half-perfect non-recent memory. Next at time t.sub.7 is where the second anomaly occurs, breaking up the linearity of non-recent memory. This is where we first begin to see the logarithmic temporality break up the twofold downsampling of non-recent memory:
1 0 0 0 1 1 1
is the result at t=t.sub.7.
[0138] Continuing until the tenth row of TIMEL MATRIX 411, at t=t.sub.10, we have perfect recent memory, and quarterspeed non-recent memory:
1 0 0 0 1 0 0 0 1 1
If we were to continue without the next anomaly, we would get:
1 0 0 0 1 0 0 0 0 1 1
and this would give rise to a non-monotonicity, i.e. a situation in which the midrange memory would actually be worse than the oldest memories. Therefore, we introduce the anomaly at time t=t.sub.11, and drop the 5th column. This drops our non-recent memory to eighth speed which we continue to time t=t.sub.18.
[0139] At time t=t.sub.19, we'd have non-monotonicity if we didn't drop to sixteenth speed capture, thus we drop the ninth column at t.sub.19. We continue to capture at sixteenth speed to t.sub.34 and then drop to 1/32 speed at t.sub.35 by dropping the 17th column.
[0140] The anomalous rows indices are simplified by stripping off the first L rows (in this case the first 4 rows since here L=4) of perfect memory, so that these row indices occur at m=1, 3, 7,15, 31, 63, and so on, i.e. for at =2.sup.z1 for some natural number (i.e. positive integer), z>0, where these rows are indicated with *** in the comments below:
TABLE-US-00002 B=8; % B=4 for TIMEL MATRIX 420, and B=8 for TIMEL MATRIX 430. M=2..{circumflex over ()}B-1+4; N=2..{circumflex over ()}B-1+4; % good sizes: M=N=2..{circumflex over ()}B+3=7,11,19,35,67, 131, etc.. a=zeros(M,N); for m=1:M; a(m,[1:m])=1; end % lower triangular atop=a(1:4,1:M); % top 4 rows are saved for later a=a(5:M,1:N); % take off top 4 rows to see pattern the easier %a( 1:M-4, 2)=0; %*** even col %a( 2:M-4, 4)=0; % even col %a( 3:M-4, 3)=0; %*** earliest odd col %a( 4:M-4, 6)=0; %a( 5:M-4, 7)=0; %a( 6:M-4, 8)=0; %a( 7:M-4, 5)=0; %*** next would result in non-semimonotonicity, so drop 5 %a( 8:M-4,10)=0; %a( 9:M-4,11)=0; % delete newer row when possible, i.e. preserve oldest %a(10:M-4,12)=0; %a(11:M-4,13)=0; %a(12:M-4,14)=0; %a(13:M-4,15)=0; %a(14:M-4,16)=0; %a(15:M-4, 9)=0; %*** next would result in non-semimonotonicity, so drop 9 for b=1:B; % log base 2 bit length for m=2..{circumflex over ()}(b1):2..{circumflex over ()}b2; % e.g. 16 to 30 a(m:M-4,m+2)=0; % non-anomalous row end%for m m=2..{circumflex over ()}b1; % *** e.g. 31 a(m:M-4,2..{circumflex over ()}(b1)+1)=0; % *** anomalous row end%for b A=[atop;a]; % put the top 4 rows back on
[0141] TIMEL MATRIX 420 is for B=4 with M=N=19 and TIMEL MATRIX 430 is for B=8 with M=N=259. Once again, these matrices are often conceptual constructs for conveying and specifying the algorithm, rather than for being stored in memory, e.g. more typically for B=16, M=N=65539 and we may not wish to store a matrix having M*N=4295360521 elements in memory, but simply implement the above algorithm without ever storing the TIMEL MATRIX.
[0142] TIMEL MATRIX 440, of which only the first 15 and last 3 rows are shown, depicts a situation with a memory element array of length, L=8. The last row depicts the situation where a state of completely logarithmic memory fidelity has been attained.
[0143] More generally, embodiments of the invention implement mixtures of memory models like linear and logarithmic, as well as mixtures of memory fade models like downsizing and down-compressing.
[0144]
Rather than trying to replace humans with computers, HI works to make human superintelligence arise naturally from quickly-responsive feedback loops.
[0147] Likewise a camera system can not merely augment human memory but also help improve natural human memory and intellect by creating the right kind of model to help people remember in a more natural way.
[0148] The natural way in which human memory works is to remember more recent occurrences with full fidelity, after which memory degrades gracefully.
[0149] Electronically captured memory exists on Memorylines, such as ancient memoryline 572, or more recent memoryline 590. The memoryline is a timeline of images like the timeline in a movie editor program, but with various resolutions, compressions, etc., along the timeline, so as to provide a fading timeline that gracefully fades off into the past. The possibility of electronically captured memory, whether captured or not, exists on Memorylanes, such as memorylane 560 which existed, for example, before the recording process began, or memorylane 561 which is an interpolated memorylane, which might, for example, be obtained by reading between the lines of the memoryline at time t.sub.14 and the memoryline at time t.sub.15.
[0150] The system of
[0151] In some embodiments of the invention, we grab small low-resolution test images of a scene at a much higher rate of capture, and then, based on automated image analysis, decide when to capture at full resolution. In some embodiments the low resolution images are simply just a few thousand pixels or a few pixels or even just one pixel, or one average light level, or they derive from another sensor input like a sound sensor (e.g. a microphone is used as a gunshot detector). Thus, more generally, holdings are collections of gettings, where a getting is the capture of data which can be image data or other data that we use for automation of image data capture. In some embodiments, sensors work in confluence, e.g. the gunshot detector marks images as having higher visual saliency. Other visual saliency indicators include brainwave sensors, heart sensors, and other physiologicals, done by wearable apparatus, or by remote sensing like radar, video (Eulerian Video Magnification or the like), etc. In some embodiments, visual saliency is emotion, along the lines of Rosalind Picard's work on Affective Computing. Generally there is a steady-state memory fade, punctuated by indicators of interest like visual saliency.
[0152] A useful embodiment of the invention uses these low-resolution image captures to predict exposure trends to get smoothly varying flicker-free timelapse pictures, with lightspace management. For example, images are captured at a high frame rate and low spatial resolution and analyzed for cloud cover, or the like, to guide the capture of high resolution images to make them more similar to each other thus reducing lightspace flicker. In one embodiment the image capture times are adjusted slightly to bias the capture toward identical lighting, such as lighting where the sun is shining more (or not shining as much), to match previous image captures where the sun was shining more (or less). More generally, exposure and fill flash are adjusted automatically to result in timelapse smoothing, to eliminate timelapse flicker.
[0153] Storing these small often very low-resolution test images takes very little memory, and helps to dot the memoryline with extra information that is useful in reconstructions and interpolations.
[0154] Likewise, frames never need to be totally deleted; they are downsampled, downsized, downgraded, downconverted, downcompressed, or the like. In some embodiments, the downgrading is done by removing random pixels rather than uniform downsizing, so as to facilitate Compressed Sensing reconstruction. In other embodiments, coefficients of transform compression are downscaled according to the less important coefficients of a transform encoding.
[0155] At time t.sub.8 the memoryline 572 is full. Capturing a new frame at time t.sub.9 would normally results in eviction of the second holding element of memoryline 573, but instead of deleting the second holding of the memoryline, it is downgraded, to a reduced resolution, reduced dynamic range, and reduced image compression quality, denoted by the smaller rectangle (to denote smaller size) and by a dotted line to denote more pixelation and more harshly quantized Huffman coding tables in transform-based image encoding.
[0156] At time t.sub.10, the second holding remains downgraded. Once a holding is downgraded the data is permanently lost. Thus if the recording stopped at time t<t.sub.8, wed recover the second holding at full resolution, but at time t>t.sub.8 we'd only get the second holding at reduced resolution, quality, fidelity, etc. At time t.sub.10, the fourth holding is downgraded, in a way similar to the way that the second holding was downgraded at time t=t.sub.9. At time t=t.sub.11, the sixth holding is similarly downgraded. At time t=t.sub.12, the eighth holding is similarly downgraded. This results in loglin (logarithmic/linear) full memoryline 575, with full resolution and full image quality for ever other frame going right back to the beginning, and every frame of recent memories 559.
[0157] A downgrader is a device (whether implemented by hard, soft, or firm ware) that accepts a full fidelity image as input, and produces a downgraded image as output. Downgraders for the above second, fourth, sixth, and eighth holdings, form a secondorder resparsifier 580, i.e. one that has a slope of 2 on the Mem.holding versus Time axes. This resparsifier 180 slope defines a (re)sparsifier schedule in resparsifier 580: downgrade every second image until caught up with but not beyond the recent memories 559.
[0158] In this embodiment we desire some portion of memory, recently recorded, such as recent memories 599, to be recorded with perfect fidelity. Thus at time t=t.sub.13, resparsifier 580 stands down, and does not continue onwards to downgrade the tenth holding. Instead, resparsifier 581 downsizes the third holding, and midsizes the seventh holding, so that every fourth frame is retained at full quality and resolution, and odd frames are retained at full quality but moderate resolution along memoryline 576. The seventh holding, holding h.sub.7, receives its first data at time t=t.sub.7, and begins being resparsified at time t=t.sub.13. At time t=t.sub.14, the 14th holding h.sub.14 receives its first data, such that recent memories 599 comprise frames captured at times t=t.sub.11, t.sub.12, t.sub.13, and t.sub.14, and resparsifier 580 comes back to life to resparsify the tenth holding, so that only every fourth frame of non-recent memories is retained at full resolution and quality. There are three such full frames of non-recent memory, so that the memoryline at t.sub.14 has only seven frames at full fidelity. For nonrecent memory, the in-between frames comprise downsized memories 551 and midsized memories 552. In this way, there's a reasonable amount of fidelity for every-other-frame (i.e. for all the odd-numbered frames) of non-recent memory. Notice that the memoryline at time t=t.sub.14 has some properties of linearity and some properties of logarithmicarity, i.e. the seventh holding is kept at better fidelity than the third holding, thus favouring recent memories over older memories, but still retaining older memories to some degree, as the human mind does.
[0159] At time t=t.sub.15, a new frame arrives to the fifteenth holding, and to make room for it, the third and seventh holdings are further downgraded. At time t=t.sub.16, the twelfth holding is downcompressed to join the ranks of downcomp memories 553, and the third and seventh holdings are further downgraded.
[0160] During this process of gracefully forgetting the past, there are certain ancient memories 550 that remain clear. In this embodiment, the first frame is kept at full fidelity, so that we have at least some ancient full fidelity in the oldest memories. But some of the oldest memories that are being downgraded, become further downgraded as eroding memories 554.
[0161] In some embodiments, the eroding memories 554 fade out in an approximately logarithmically decaying image resolution, while rolling off also in bitrate (compression versus quality).
[0162]
[0163] Thus we can understand television as a rasterized timescale, where the temporal frames are at low frequencies, the rasters or rows at mid frequencies, and the individual picture elements on the screen at high frequencies.
[0164] Like television, there are many other periodic phenomena in life. Another example is the movement of celestial bodies like the sun:
Tired of lying in the sunshine . . .
So you run and you run to catch up with the sun but it's sinking
Racing around to come up behind you again.
The sun is the same in a relative way but you're older,
Shorter of breath and one day closer to death. [0165] Pink Floyd, Time, Mason, Waters, Wright, Gilmour, The Dark Side of The Moon
[0166] Each day the sun moves through the sky, creating a sequence of images that each have a particular set of shadows, for each time-of-day. Each morning, the shadows run from east to west. Each afternoon they run from west to east.
[0167] Images are therefore grouped by time-of-day, and by date, into a two-dimensional array, as shown in
[0168] Thus memorylines 670 comprise both timelines and datelines. In this sense there is a memory array that represents an attempt at organizing the data into a multidimensional array; in
[0169] The days in June are longer than the days in January or December, and in fact the longest day is typically June 21, where the sun rises earlier and sets later, whereas on the shortest day typically December 22, the sun rises later and sets earlier.
[0170] Accordingly, in some embodiments, we re-sample the data (interpolate) to resynthesize datasets in which each row or column represents shadows from the sun at a particular azimuth. In other embodiments of the invention, capture itself is scheduled to correspond to the sun's azimuth, e.g. one picture for each degree or each five degrees of sun's movement, or the like.
[0171] Thus we have lightspaces, giving rise to lightvectors like lightvector v.sub.1 that represents all the pictures taken at sunrise, throughout the year. Not every day is sunny, but some of them are. So a good number of the 365 days of the year, there will be sunny day sunrises that capture the subject matter with good clear long shadows of sunrise. Late morning toward noon, we might have for example, lightvector v.sub.5, say, for example, all the pictures captured at high noon. Lightvector v.sub.10 represents all the pictures captured at sunset. These lightvectors define memorylanes like memorylane 661 the defines sunrise pictures, memorylane 665 that defines noon pictures, and memoryline 6610 that defines sunset pictures.
[0172] Organizing or capturing the data in this way defines lightvectors 650 that are used to separate azimuth from elevation. In this way such timelapse image capture can be used to generate inverse holograms (e.g. the margoloh, as defined in Recording Lightspace so shadows and highlights vary with varying viewing illumination, by S. Mann, in Optics Letters, Vol. 20, Iss. 4, 1995).
[0173] Specifically,
[0174]
[0175] Let us suppose for our simple example here that we only have enough memory to store two recordings (i.e. two frames of data such as audio records). Thus when a second recording arrives at time t.sub.2, we resparsify the first recording so that it takes up half the space. For example, we throw away every second sample so it only has 16 samples in it instead of 32. Preferably, though, we resparsify its coefficients in a transform space, thus keeping all the samples (to maintain the Nyquist sampling criterion), and only suffering a slight loss in quality.
[0176] At time t3, when a new recording arrives, we resparsify all the old recordings (all two of them) to half their size, to make ample room for the third recording. When a fourth recording comes in at time t.sub.4, we resparsify all three of the older recordings again by half, and so on. Thus holdings h.sub.1, h.sub.2, h.sub.3, and so on, along holdlines 701, decay exponentially, as we progress along timelines 700.
[0177] Our plan is simple: each time a new recording arrives, we resparsify everything else we've got, for being down by a factor of two in storage space.
[0178] The amount of space that we will need, if we keep doing this for an infinitely long time, is:
C=1+++++ 1/32+ 1/64+ . . . =2.(1)
[0179] Thus we can record for an infinite duration into a finite amount of memory, i.e. into we can capture an infinite number of audio recordings into a space that only has sufficient memory for two audio recordings.
[0180] In some embodiments, we wish to mimic human memory, such as so-called Flashbulb Memory, e.g. the way in which people remember all the details of what happens at the time of a significant event. For example, most people old enough to remember the assisination of President Kennedy remember a lot of seemingly minute details of the environment around them when they first heard the news. They often remembereven many years laterthe paint colour on the walls in the room they were in when they first heard the news, and the minute details of the designs on the wallpaper, and even which foot was in front of the other foot while they were walking, when the suddenly stopped in shock at the news.
[0181] Suppose that at time t.sub.5, some highly significant event occurs, such as a gunshot, as detectable by a gunshot detector in the apparatus of the invention, or by other inputs used with the invention, such as a brainwave sensor or electrocardiographic sensor, shock sensor, vibration sensor, earthquake sensor, seismic sensor, gunfight sensor, or visual saliency sensor (as in the above-mentioned Intelligent Image Processing textbook).
[0182] The new sample comes in at time t=t.sub.5, and then when the next sample comes in at time t=t.sub.6, in order to make room for it, we simply delete all previous samples, so we now have only the two samples: the current sample at t.sub.6, and the (e.g. gunshot) sample at t.sub.5.
[0183] In some embodiments of the invention, it will keep only the gunshot sample and the current sample, and current sample, i.e. remain always thereafter at a simple capacity as:
C=1+1=2.(2)
But more often, we wish to continue to be able to remember things.
[0184] So when the next sample arrives at time t=t.sub.7, we instead resparsify the two previous recordings to half their storage capacity, thus using now a total storage capacity at t=t.sub.7 of:
C=1+++ . . . =2.(3)
[0185] At time t=t.sub.8 when the next sample comes in, we wish to preserve the t.sub.5 sample at its present value, i.e. prevent it from going down below occupying one half the storage space of one full recording.
[0186] To accomplish this, we take the two recordings therebetween, which would normally total 1+ a recording's worth of data, and drop these to a further of their size. This results in:
C=1+(1+)+=1++=2.(4)
[0187] We apply this rule recursively, resparsifying the recordings between the current recording and the gunshot recording, by each time.
[0188] In this way we always remember what has happened since the gunshot occurred, but we have a permanent aural memory impairment due to the gunshot effect, as if slightly deafened by the gunshot, so that now our sound memories drop off by each time rather than only by each time.
[0189] So at t=t.sub.9 we have:
C=1+(1+(1+))+=1++=2(5)
[0190] Then at t=t.sub.10 we have:
C=1+(1+(1+(1+)))+=1++=2,(6)
and so on, always being able to record for an infinite duration into a finite memory, while preserving to a very good degree the gunshot recording.
[0191] This is a very extreme and simple example, where the permanent impairment is quite profound in its effect on memory, but more typically, we have more memory than used in this example. We might for example have enough memory for a few months worth of recording space, and thus be able to record infinitely and still remember many different important events very well while still not suffering such a strong impairment.
[0192] Also, as hard drive costs go down and camera resolutions increase, we have embodiments that allow hot-swappable storage without loss in recordings, and in fact each time the storage is increased, we get a reprieve on resparsifications, for a little while, as we expand into larger space. Likewise as camera resolutions increase, we may upgrade our device while the recording is happening, without interruption thereof. In this way we get infinite recordings in finite memory that stop decaying each time we upgrade.
[0193] For example, if the new camera or new audio recorder or device arrives at time t=11 and it has twice or three times the resolution, e.g. in this simple example, maybe 64 samples per recording, and its hard drive is bigger, e.g. twice or three times as big, we don't need to resparsify because the old recordings will look already small in light of the new standards of resolution.
[0194] Indeed, old 640480 recordings don't need to be downgraded when we transition into an HDTV world or 4k video world, because the old recordings already then at that time t=t.sub.11 pale in comparison to the whole scale of things at the new time.
[0195] So the invention allows for SCALEABLE infinite recording into finite but growing memory.
[0196] Michel Foucault, in his seminal book, Surveiller et Punir, outlined much of the work on surveillance as of the era of Jeremy Bentham's Panopticon, which brought with it the birth of the modern prison, and of the modern carceral society, Carceral archipelago, and Prison Planet.
[0197] The English translation of the book was published as Discipline and Punish, where the notion of Surveillance translates to the notion of Discipline, i.e. not merely sensing but also effecting.
[0198] To be punished through discipline is to suffer, and the opposite of to suffer to do. When we think of control theory, we have observability and controllability. Surveillance is often used as a means of control of a society, from top-down.
[0199] We live in a world of surveillance, but it doesn't have to be only that way. We aim to create not a surveillance-society, but a veillance society.
[0200] Surveillance means watching from above (its a French word that translates roughly to Oversight in English). Veillance is simple concept==Sight==neither from above nor from below. Veillance is a fair and balanced sight, that includes components of surveillance, sousveillance (undersight), coveillance, and more importantly, OpenVeillance!
[0201] But Veillance is more than just sight. Just as surveillance includes also the hidden microphones and wiretaps of audio conversations, Veillance also includes audio as well as video as well as other sensory dimensions.
[0202] And just as Surveillance includes both observability and controllability (i.e. to Discipline is not just to sense but also to effect, i.e. both observability and controllability! In this way, Veillance is the more open, fair, and balanced form of control theory through technologically open means.
[0203] Just as we have Compressed Sensing, we can also have Compressed Affecting. Compressed Sensing is to observability as Compressed Affecting is to controllability, thus giving rise to the feedback loop of Humanistic Intelligence (See Mann 1998, Proc. IEEE Volume 86, Number 11, entitled Humanistic Computing).
[0204] As machines are more intricately woven into the fabric of our everyday life, machine intelligence presents existential risks to humankind [Markoff 2015, Bostrom 2014], and we must guard against these risks. But survival of the human race is not enough. We need to ensure we're not living a degraded and debased life under the surveillance of intelligent machines that want to know everything about us and yet reveal nothing about themselves. Take for example television. Early television sets tried to display a picture no matter how bad the signal: It simply did its best to show us a picture at all times, even if that picture was flawed or contained some random noise or optical snow. A modern television receiver only allows its owner to see a picture if the television decides it is crisp enough to be approved for viewing. Otherwise, the TV set just displays a blue screen with the message No Signal. Without continuous feedback, we can't move the antenna a little bit, or wiggle the wires, or quickly try different settings and inputs, to see what improves the picture. Machines have become like great gods, made perfect for worship, and are no longer showing us their flaws that used to allow us to understand them. In a sense the human has been taken out of the feedback loop, and we no longer get to see (and learn from) the relationship between cause (e.g. the position of a TV antenna or wiring) and effect (e.g. the subtle variation in picture quality that used to vary continuously with varying degree of connectivity).
[0205] More seriously, as our computers, web browsers, and other things, become filled with opaque and mysterious operating systems that claim to be helpful, they actually reveal less about how they work. Like the television with No Signal, when our web browser says Page not found, in a world of secret technologies, we now know less and less about how our sources of information might be corrupted. Increasingly, the human element in our democracypublic opinioncan be manipulated, without the public knowing why. It all comes back to a simple question: whether we allow technology to be opaque and closed-source, or whether we force it to have the openness of integrity.
[0206] In all of these, what we've lost is observability, which is a measure of how well the internal states of a machine can be inferred from its external outputs [Kalman 1960].
[0207] Machines have inputs and outputs, as do humans. Together that's four in/out paths: Machine in; Machine out; Human in; Human out. But when the human and machine operate together with feedback, there is also Observability and Controllability, which add two more, for a total of six paths, giving rise to a Humanistic Intelligence.
[0208] As we build machines that take on human qualities, will they become machines of integrity and loving grace==machines that have the capacity to love and be loved, or will they become machines of hypocrisy==one-sided machines that can't return the love and trust we give to them. If machines are going to be our friends, and if the machines are going to win our trust, then they also have to trust us. Trust is a 2-way street. So if the machines don't trust us and if the machines refuse to show their flaws to us, then we can't trust them. Machines need to show us their imperfections and their raw state, like a good friend or spouse where you trust each other and show yourselves, e.g. naked or unfinished. If machines are afraid to be seen naked or unfinished, then we should not show ourselves in that state to them. Technology that is not transparent cannot be trusted.
[0209] Indeed, when machines want to know everything about us, yet reveal nothing about themselves, that's a form of hypocrisy.
[0210] The opposite of hypocrisy is integrity. Thus machines observing us, and not themselves observable, are machines that lack integrity.
[0211] Observability and integrity require responsiveness (quick immediate feedback) and comprehensibility (making machines that are easy to understand). Systems implementing HI help you understand and act in the world, rather than masking out failure modes as do TV standards like HDMI. Indeed we're seeing a backlash against the machines as gods dogma, and a resurgence of the glitchy raw aspects of quick responsive comprehensible technologies like vacuum tubes, photographic film, and a return to the steampunk aesthetic and incandescent light bulbs (transparent technology).
[0212] Machines are driving us insane and making us stupid.
[0213] Consuer a Big Data and Little Data framework as outlined in the following table:
TABLE-US-00003 Big Data Little Data Artificial Intelligence Humanistic Intelligence [link] Internet of Things Wearables [link] Smart Things (that sense people) Smart People (self sensing) Surveillance Sousveillance [link] (Oversight) (Undersight) Security Suicurity [link] Secrecy and Panoptic Privacy Openness and Genuine Privacy Hypocrisy (Half-Truth) [link] Integrity (Whole-Truth) [link] Software (typ. closed source) Computer programs (e.g. GNU) Anti-circumvention laws Tinkering as a form of inquiry
[0214] Research Priorities for Robust and Beneficial Humanistic Intelligence:
[0215] Analogous to the research priorities identified in Max Tegmark's AI Open Letter, a number of important research priorities can be identified for Humanistic Intelligence (HI). This more wholistic framework gives rise to six signal flow paths that more generally define six new human rights for a person using HI. The technologies include: [0216] Videscrow (Escrow of video and other senses); [0217] Priveillance (privacy and veillance); [0218] NotRecord (computionally perfect sensory memory without recording); [0219] Suicurity (counterpoint to security): see Mann, S. (2014). Personal Safety Devices Enable Suicurity IEEE T& S (Technology and Society), 33(2), pp 14-22; [0220] Subjectright (counterpoint to copyright); [0221] Optimum insanity/Lunatic.
[0222] Lightspace is a tensor outer product of sensing (lightfields) with affecting (time-reversed lightfields), and more generally, Compressed Control is Compressed Sensing together with Compressed Affecting.
[0223] One development from this work is the concept of Optimum Insanity. Insanity is defined as doing the same thing over and over again
[0224] There are so many software products that give us a headache and dont really work well. One of the things that weve noticed is that when things dont work, people keep trying the same thing over and over again, expecting a different result. If software doesnt work, the manufacturers or help lines or IT experts tell customers or users or clients to run it again or reboot and try again. Thats the definition of insanity: doing the same thing over and over again, expecting a different result. So insanity is the new norm. Insanity is in fact a required attribute of the software world.
[0225] For a sane person to use the invention, there is a desire to support the four attributes of memory in the Declaration of Veillance, a declaration of the rights and responsibilities of remembering things. See Declaration of Veillance (Surveillance is Half-Truth) by Steve Mann, Ryan Janzen, Mir Adnan Ali, and Ken Nickerson, Proc. IEEE GEM 2015, October 2015, Veillance Foundation, 330 Dundas Street West, Toronto, Ontario, Canada M5T 1G5. Therein the four freedoms of veillance memory are being able to See; Understand; Remember; and Share or describe those memories.
[0226] One application for recorded memory is Lunatic. Its an insanity embodying application. Its a computer program a user in order to download memories that can run that does the same thing over and over again until it gets a different result. Its like a front end to other software that allows users to click on it once and then itll keep clicking until it gets the result.
[0227] The problem is, a lot of existing hardware has software in it and software is often garbage. Thusly, a lot of the hardware has garbage in it, which is not only driving people insane, but is requiring and demanding insanity of us.
[0228] Lunatic works as follows: a memoryfile is provided for download. A user running Lunatic can click on a file to download, and Lunatic will determine (by machine learning) the Optimum Insanity with which to download the file. The Optimum Insanity is the optimum number of times to spawn the same request multiple times. Typically the Optimum Insanity is 2 or 3 times. Typically, for example, if we're trying to download a video off the Internet, there will come a time now and again when the download takes a really long time. Like a really small file that should download in one minute sits there for an hour doing nothing and if you leave it it will sit for several days doing nothing. In those cases, when you open a new tab and download the same thing again, it often comes in just the one minute.
[0229] So instead, every time you click to download a file, Lunatic tries twice in separate threads, to download the same movie or image or timelapse video file.
[0230] Then whichever one is successful first, causes the other download to be aborted because it is not needed.
[0231] Over time, if this still doesn't work, i.e. if two downloads were issued and both of them happen to get stuck, the Optimum Insanity is increased to three. Over time the system learns if this works, and then backs off re-adjusting the Optimum Insanity back down to two, if that works better.
[0232] In some embodiments of Lunatic, whenever the Optimum Insanity gets to 8, the 9th thread is run on another remote computer. When it gets to 64, the 65th thread is run on a remote computer in another country. and so on.
[0233] One effect of Lunatic is to encourage providers to fix small bugs that are leading to the Insanity inherent in software, especially Closed Source software.
[0234] Lunatic is not a Denial of Service (DoS) attack. Infact Lunatic is a Demand for Service (DfS). Lunatic is hopefully something that will only be needed in the short term, until software can be made that no longer requires insanity to use it.
[0235] From the foregoing description, it will thus be evident that the present invention provides a design for a LightSpaceTimeLapse camera or processor or service or system, or a similar system, means, apparatus, or the like. As various changes can be made in the above embodiments and operating methods without departing from the spirit or scope of the invention, it is intended that all matter contained in the above description or shown in the accompanying drawings should be interpreted as illustrative and not in a limiting sense.
[0236] Variations or modifications to the design and construction of this invention, within the scope of the invention, may occur to those skilled in the art upon reviewing the disclosure herein. Such variations or modifications, if within the spirit of this invention, are intended to be encompassed within the scope of any claims to patent protection issuing upon this invention.