CAMERA WITH PANORAMIC SCANNING RANGE
20230262337 · 2023-08-17
Inventors
- Roy Fridman (Tel Aviv, IL)
- Gil Bachar (Tel-Aviv, IL)
- Ruthy Katz (Tel-Aviv, IL)
- Itamar Zinger (Tel Aviv, IL)
- Paz Ilan (Tel Aviv, IL)
Cpc classification
G02B13/06
PHYSICS
H04N23/698
ELECTRICITY
H04N23/55
ELECTRICITY
G03B17/17
PHYSICS
G02B27/642
PHYSICS
G02B27/644
PHYSICS
H04N23/58
ELECTRICITY
H04N23/69
ELECTRICITY
International classification
H04N23/698
ELECTRICITY
G03B17/17
PHYSICS
H04N5/262
ELECTRICITY
H04N23/55
ELECTRICITY
Abstract
Cameras with panoramic scanning range comprising a folded digital camera in which an optical path folding element (OPFE) that folds a first optical path from an object or scene into a second optical path substantially parallel with an optical axis of a lens of the folded camera, the OPFE being rotatable around the lens optical axis, and systems incorporating such cameras.
Claims
1. A system, comprising: a) a dual-aperture camera that includes an upright camera with a first field of view FOV.sub.1 and a folded camera with a second field of view FOV.sub.2, the folded camera having a lens with a lens optical axis and an optical path folding element (OPFE) that folds a first optical path from an object or scene to a second optical path, wherein the second optical path is substantially parallel with the lens optical axis and wherein the OPFE is rotatable around the lens optical axis for movement of FOV.sub.2; and b) a processing unit configured to track an object or subject of interest in FOV.sub.1 to obtain tracking information and to use the tracking information to direct the OPFE to rotate for the movement of FOV.sub.2.
2. The system of claim 1, wherein FOV.sub.1>FOV.sub.2.
3. The system of claim 1, wherein the processing unit is configured to track the object or subject of interest in FOV.sub.1 autonomously.
4. The system of claim 1, wherein the OPFE is rotatable in an angle of up to 180 degrees.
5. The system of claim 1, wherein the OPFE is rotatable in an angle of up to 360 degrees.
6. The system of claim 1, wherein the OPFE being rotatable around the lens optical axis provides a plurality of different images.
7. The system of claim 1, operative to record a video stream with a changing or adaptive field of view.
8. The system of claim 1, further comprising an actuator for rotating the OPFE around the lens optical axis.
9. The system of claim 8, wherein the actuator includes a step motor or a voice coil motor.
10. The system of claim 1, further comprising a folded camera image sensor, wherein the lens is fixedly attached to the OPFE and is rotatable together with the OPFE around the lens optical axis relative to the image sensor.
11. The system of claim 10, wherein the lens is a folded lens.
12. The system of claim 1, further comprising a folded camera image sensor, wherein the lens has an image circle.
13. The system of claim 12, wherein the image sensor is smaller than the image circle.
14. The system of claim 13, wherein the image sensor is larger than the image circle.
15. The system of claim 1, wherein the tracking information is used to define an order in which images are captured with the folded camera to form a panoramic image.
16. The system of claim 2, wherein the tracking information is used to define an order in which images are captured with the folded camera to form a panoramic image.
17. The system of claim 3, wherein the tracking information is used to define an order in which images are captured with the folded camera to form a panoramic image.
18. The system of claim 10, wherein the tracking information is used to define an order in which images are captured with the folded camera to form a panoramic image.
19. The system of claim 11, wherein the tracking information is used to define an order in which images are captured with the folded camera to form a panoramic image.
20. A mobile electronic device comprising a system of claim 1.
21. The mobile electronic device of claim 20, wherein the mobile electronic device is a smartphone.
22. A method comprising: a) providing a folded digital camera that includes an image sensor, a lens having a lens optical axis and an optical path folding element (OPFE) that folds a first optical path from an object or scene to a second optical path, wherein the second optical path is substantially parallel with the lens optical axis, and wherein the folded digital camera has an original orientation; b) rotating the OPFE around the lens optical axis relative to the image sensor in a first rotation direction to set the first optical path in a desired first direction; and c) taking an image.
23. The method of claim 22, further comprising digitally rotating the taken image back to the original orientation.
24. The method of claim 23, wherein the rotating the OPFE around the lens optical axis in a first rotation direction to set the first optical path in a desired first direction includes rotating the OPFE to set the first optical path in a plurality of desired first directions in a first range of up to 180 degrees, and wherein the taking an image includes taking an image at each direction of the plurality of desired first directions, thereby obtaining a matching first plurality of taken images.
25. The method of claim 24, further comprising constructing a first panoramic image from the first plurality of taken images.
26. The method of claim 24, further comprising rotating the OPFE around the lens optical axis in a second rotation direction opposite to the first rotation direction, to set the first optical path in a plurality of desired second directions in a second range of up to 180 degrees opposite to the first range, and wherein the taking an image includes taking an image at each direction of the plurality of desired second directions, thereby obtaining a matching second plurality of taken images.
27. The method of claim 26, further comprising constructing a second panoramic image from the first plurality of taken images.
28. The method of claim 27, further comprising combining the first and second panoramic images into a combined panoramic image.
29. The method of claim 23, wherein the lens is fixedly attached to the OPFE and wherein the rotating the OPFE around the lens optical axis relative to the image sensor in a first rotation direction includes rotating the lens together with the OPFE.
30. The method of claim 23, wherein the lens is a folded lens.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Non-limiting examples of embodiments disclosed herein are described below with reference to figures attached hereto that are listed following this paragraph. Identical structures, elements or parts that appear in more than one figure are generally labeled with a same numeral in all the figures in which they appear. The drawings and descriptions are meant to illuminate and clarify embodiments disclosed herein, and should not be considered limiting in any way.
[0035]
[0036]
[0037]
[0038]
[0039]
[0040]
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049]
[0050]
[0051]
[0052]
[0053]
[0054]
[0055]
[0056]
[0057]
[0058]
[0059]
[0060]
[0061]
[0062]
[0063]
DETAILED DESCRIPTION
[0064] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well-known methods have not been described in detail so as not to obscure the presently disclosed subject matter.
[0065] It is appreciated that certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination.
[0066] The term “processing unit” as disclosed herein should be broadly construed to include any kind of electronic device with data processing circuitry, which includes for example a computer processing device operatively connected to a computer memory (e.g. digital signal processor (DSP), a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.) capable of executing various data processing operations.
[0067] Furthermore, for the sake of clarity the term “substantially” is used herein to imply the possibility of variations in values within an acceptable range. According to one example, the term “substantially” used herein should be interpreted to imply possible variation of up to 10% over or under any specified value. According to another example, the term “substantially” used herein should be interpreted to imply possible variation of up to 5% over or under any specified value. According to a further example, the term “substantially” used herein should be interpreted to imply possible variation of up to 2.5% over or under any specified value.
[0068] In the text below, “digital rotation” is used to describe an image rotation by software, to distinguish from just “rotation” used to describe physical rotation of an optical element.
[0069]
[0070] In some embodiments, camera 100 may further include a focus or autofocus (AF) mechanism (not shown), allowing to move (or “shift” or “actuate”) lens 102 along axis 110, such that is can focus images from objects at various distances on image sensor 106. For simplicity, the description continues with reference only to AF, with the understanding that it also covers regular (manual) focus. The AF actuation mechanism is typically of a voice coil motor (VCM) type, i.e. a “VCM actuator”. Such actuation mechanisms are known in the art and disclosed for example in Applicant's co-owned international patent applications PCT/IB2015/056004 and PCT/IB2016/055308. This is however a non-limiting example, and the AF mechanism may be of other types, such as a stepper motor, a shape memory alloy (SMA) actuator, or other types known in the art. In some embodiments, camera 100 may include an optical image stabilization (OIS) actuation mechanism (not shown) in addition to, or instead of, the AF actuation mechanism. OIS may be achieved for example by shifting the lens in two directions in the X-Y plane, compensating for tilt of camera 100 around Z and X directions. A three degrees of freedom (DOF) OIS+focus actuation mechanism (which performs two movements for OIS and one for AF) is typically of VCM type and known in the art, for example as disclosed in international patent application PCT/US2013/076753 and in US patent application 2014/0327965. More information on auto-focus and OIS in a compact folded camera may be found in Applicant's co-owned international patent applications PCT/IB2016/052143, PCT/IB2016/052179 and PCT/IB2016/053335.
[0071] In contrast with known folded camera modules (see e.g. PCT/IB2016/052179) camera 100 is designed to rotate OPFE 104 around axis 110 (the Z axis) relative to the image sensor, i.e. in the X-Y plane in the coordinate system shown, a rotation indicated by an arrow 112. OPFE 104 can rotate in an angle range as required by optical requirements (see below), in some cases by up to 180 degrees and in other cases by up to 360 degrees.
[0072] In some embodiments, lens 102 may be optically axisymmetric. Therefore, any rotation of lens 102 around axis 102 does not change any optical property of the system and in particular the image. In such embodiments, lens 102 may rotate together with OPFE 104. In particular, as shown in an exemplary embodiment in
[0073] The rotation of OPFE 104 around axis 110 relative to the image sensor by “a” degrees will cause axis 108 (which, in its original state before rotation, is positioned perpendicular to the X axis in the coordinate system shown) to rotate in the X-Y plane and will result in two changes in the image on the image sensor: a) rotation of the center field-of-view (FOV) by a degrees and b) rotation of the image on image sensor (known in the art as “Roll” effect) by a degrees.
[0074] The rotation of the OPFE as described above and the consequent rotation of the first optical path allows photography of a panoramic view. Camera 100 has a panoramic scanning range. The panoramic view (and the scanning range) may be of up to 360 degrees. A plurality of photographs also referred to below as “sub-views”, “image sections” or “view sections”, each sub-view reflecting or related to a particular OPFE rotation positions, may be acquired and “stitched” into a “panoramic output image”.
[0075] An “image circle” of lens 102 (see also
[0076]
[0077]
[0078]
[0079] In another embodiment, camera 100 may be used to a take video, or a preview of a video on an operating device screen (see below operating devices examples). The scanning capability allows selection of the video field of view. An example below (in
[0080] Several non-limiting examples (for example a smartphone and a flying drone) of platforms carrying or including a system such as camera 120 with camera 100 are presented in
[0081]
[0082] In some embodiments, a folded camera with panoramic scanning range disclosed herein may be positioned together with a non-folded (“upright”) non-scanning camera in a dual-aperture (or “dual-camera”) camera arrangement. The non-folded camera may have a FOV smaller than, equal to or larger than the FOV of the folded camera. Dual-aperture cameras including a folded camera and a non-folded camera are described for example in PCT/IB2015/056004. The dual camera may be positioned in a smartphone or in other personal electronic devices. The upright camera may be either a front camera or a back camera of the smartphone.
[0083]
[0084]
[0085]
[0086]
[0087]
[0088]
[0089] In TV set 800, cameras 812 and 830 are located on a top left corner of TV set 800. In other exemplary embodiments, cameras 812 and 830 may be located in other positions, such as the top center, the left or right side, the bottom side or even beyond screen 802 described below. In another exemplary embodiment, cameras 812 and 830 may be located in a separate module (box) outside of the TV set, connected via cable or via cordless connection.
[0090] In an exemplary use embodiment, TV set 800 may be used for video-conferencing as follows: a first user (person) located in a first location may use TV set 800 to communicate with a second user (person) located in a second location. The second user may use any electronic device comprising a screen, for example a TV, a smartphone, a personal computer, a notebook or laptop computer, etc. In an exemplary embodiment, cameras 830 and 812 may be used to video record the first user, while the second user may see recordings from both cameras 830 and 812 on his screen.
[0091] TV set 800 may be used for automatic movement or “automatic adjustment” of FOV.sub.T for e.g. tracking a subject in an autonomous manner. A camera mode that performs automatic Tele FOV movement to track an object or subject of interest is referred to herein as autonomous Tele FOV tracking”. An exemplary autonomous tracking system and method applicable herein is described in PCT/IB2016/057366 for a smartphone system. The autonomous FOV.sub.T movement is in response to recognition (through e.g. camera 812) of the object or subject of interest, and the Tele image focuses on and displays the object or subject of interest. The object recognition may be performed using any of the methods known in the art.
[0092] An example of autonomous FOV.sub.T tracking scenario using TV set 800 is shown in
[0093]
[0094] In the particular example shown, one sees side-by-side views of video streams from both cameras: column (a) shows a screen 900a with FOV.sub.W and FOV.sub.T seen by the first user, while column (b) shows on the right a screen 900b of the second user with FOV.sub.W and FOV.sub.T and on the left a magnified Tele view (image) of part of the face and body of first user 902 and part of the background. In general, the FOV.sub.T in (a) may show the first user (as shown), the second user or another feature from the scene. The FOV.sub.T in (a) showing the first user should therefore not be considered as limiting. In general, the second user (not shown) may see on his screen a video stream from either camera 812 or camera 830, or both cameras 812 and 830 simultaneously (e.g. side-by-side). In case the second user sees only one steam of video, switching between streams may be done using smooth transition techniques, as described for example in co-owned U.S. Pat. No. 9,185,291.
[0095] The decision to track the first user may be taken by the first user, the second user (e.g., by remote control usage) or automatically (e.g., using face detection). It is assumed that the Tele camera can change its FOV by rotating the prism to track the object of interest, as disclosed above. Ideally, the camera will track the object such that it is as close as possible to the center of the adjustable FOV.sub.T as seen in the left side of (b) in each figure.
[0096] Video streams from cameras 812 and/or camera 830 may be recorded for later usage, which may include additional processing such as image processing, video stream blending, etc.
[0097] While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. For example, while a camera and a folded camera with panoramic scanning range are described as exemplarily incorporated in smartphones, flying drones and television sets, such cameras and folded cameras may be also incorporated in other platforms such as vehicles, or incorporated in platforms other than smartphones, for example tablets, laptop computers, phablets, desktop computers, smart speakers, smart watches, electronic book readers, smart glasses, smart helmets, baby monitors, augmented reality systems, virtual reality systems, advanced driving assistance systems (ADAS), etc. The disclosure is to be understood as not limited by the specific embodiments described herein, but only by the scope of the appended claims.
[0098] Unless otherwise stated, the use of the expression “and/or” between the last two members of a list of options for selection indicates that a selection of one or more of the listed options is appropriate and may be made.
[0099] It should be understood that where the claims or specification refer to “a” or “an” element, such reference is not to be construed as there being only one of that element.
[0100] All references mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual reference was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention.