Dual-aperture zoom digital camera with automatic adjustable tele field of view
11392009 · 2022-07-19
Assignee
Inventors
- Gal Shabtay (Tel-Aviv, IL)
- Ephraim Goldenberg (Ashdod, IL)
- Eran Kali (Jerusalem, IL)
- Noy Cohen (Tel Aviv, IL)
- Gil Avraham (Givat Ada, IL)
- Ruthy Katz (Tel-Aviv, IL)
Cpc classification
H04N23/45
ELECTRICITY
G03B17/17
PHYSICS
H04N23/58
ELECTRICITY
H04N23/951
ELECTRICITY
G02B13/06
PHYSICS
H04N23/55
ELECTRICITY
G03B2217/002
PHYSICS
H04N23/695
ELECTRICITY
H04N23/90
ELECTRICITY
H04N23/69
ELECTRICITY
International classification
G02B13/06
PHYSICS
G03B17/17
PHYSICS
Abstract
Digital camera comprising an upright Wide camera configured to provide a Wide image with a Wide image resolution and a folded Tele camera configured to provide a Tele image with a Tele image resolution higher than the Wide image resolution, the Wide and Tele cameras having respective Wide and Tele fields of view FOV.sub.W and FOV.sub.T and respective Wide and Tele image sensors, the digital camera further comprising a rotating OPFE operative to provide a folded optical path between an object or scene and the Tele image sensor, wherein rotation of the OPFE moves FOV.sub.T relative to FOV.sub.W. In some embodiments, a rectangular FOV.sub.T is orthogonal to a rectangular FOV.sub.W. When included in a host device having a user interface that displays FOV.sub.T within FOV.sub.W, the user interface may be used to position FOV.sub.T relative to FOV.sub.W, scan FOV.sub.T across FOV.sub.W and acquire, store and display separate Wide and Tele images, composite Wide plus Tele images and stitched Tele images. The positioning of FOV.sub.T within FOV.sub.W, can be done automatically (autonomously) by continuously tracking an object of interest.
Claims
1. A method for obtaining a high-resolution image using a digital camera comprising a Wide camera that provides a Wide image with a Wide resolution and a Wide field of view (FOV.sub.W), and a folded Tele camera that provides plurality of Tele images with a Tele resolution higher than the Wide camera resolution and with a Tele field of view (FOV.sub.T) smaller than the FOV.sub.W, wherein the Wide camera includes a Wide lens with a Wide lens optical axis and a Wide image sensor with a width W.sub.W and a height H.sub.W, wherein the Tele camera includes a Tele image sensor with a width W.sub.T and a height H.sub.T, wherein W.sub.W>H.sub.W and wherein W.sub.T>H.sub.T, the method comprising: a) positioning H.sub.T in parallel with the Wide lens optical axis and W.sub.W in parallel with W.sub.T; b) capturing a first Tele image with the FOV.sub.T at a first position within the FOV.sub.W; c) detecting an object appearing in the Wide image FOV; d) moving the FOV.sub.T relative to FOV.sub.W to track the detected object.
2. The method of claim 1, wherein the moving the FOV.sub.T relative to FOV.sub.W to track the detected object includes moving the FOV.sub.T to track the object autonomously.
3. The method of claim 1, further comprising incorporating the digital camera into a host device having a user interface for operating the digital camera.
4. The method of claim 3, wherein the host device is a smartphone.
5. The method of claim 1, further comprising, with the FOV.sub.T positioned at the first position within the FOV.sub.W, capturing a first Wide image simultaneously with the first Tele image.
6. The method of claim 1, further comprising indicating the first Tele image by a frame visible on a screen of the digital camera, the frame showing and defining the FOV.sub.T.
7. The method of claim 1, wherein the moving the FOV.sub.T relative to the FOV.sub.W includes moving the FOV.sub.T relative to the FOV.sub.W to obtain a plurality of additional Tele images with the FOV.sub.T at respective additional positions within the FOV.sub.W.
8. The method of claim 7, wherein at least some of the first Tele image and the additional Tele images partially overlap each other.
9. The method of claim 7, wherein at least some of the first Tele image and the additional Tele images do not overlap each other.
10. The method of claim 7, wherein the moving is performed manually.
11. The method of claim 7, wherein the moving is performed automatically.
12. A digital camera, comprising: a) a Wide camera that provides a Wide image with a Wide resolution and a Wide field of view (FOV.sub.W); and b) a folded Tele camera that provides plurality of Tele images with a Tele resolution higher than the Wide camera resolution and with a Tele field of view (FOV.sub.T) smaller than the FOV.sub.W, wherein the Wide camera includes a Wide lens with a Wide lens optical axis and a Wide image sensor with a width W.sub.W and a height H.sub.W, wherein the Tele camera includes a Tele image sensor with a width W.sub.T and a height H.sub.T, wherein W.sub.W>H.sub.W and W.sub.T>H.sub.T, wherein H.sub.T is parallel with the Wide lens optical axis and W.sub.W is parallel with W.sub.T, and wherein the digital camera is configured to capture a first Tele image with the FOV.sub.T at a first position within the FOV.sub.W, to detect an object appearing in the Wide image FOV and to move the FOV.sub.T relative to FOV.sub.W to track the detected object.
13. The digital camera of claim 12, wherein the digital camera configuration to move the FOV.sub.T relative to FOV.sub.W to track the detected object includes a configuration to move the FOV.sub.T to track the object autonomously.
14. The digital camera of claim 12, wherein the digital camera is incorporated in a host device having a user interface for operating the digital camera.
15. The digital camera of claim 14, wherein the host device is a smartphone.
16. The digital camera of claim 12, wherein the digital camera is configured to, with the FOV.sub.T positioned at the first position within the FOV.sub.W, capture a first Wide image simultaneously with the first Tele image.
17. The digital camera of claim 12, wherein the digital camera is configured to indicate the first Tele image by a frame visible on a screen of the digital camera, the frame showing and defining the FOV.sub.T.
18. The digital camera of claim 12, wherein the digital camera configuration to move the FOV.sub.T relative to the FOV.sub.W includes a configuration to move the FOV.sub.T relative to the FOV.sub.W to obtain a plurality of additional Tele images with the FOV.sub.T at respective additional positions within the FOV.sub.W.
19. The digital camera of claim 18, wherein at least some of the first Tele image and the additional Tele images partially overlap each other.
20. The digital camera of claim 18, wherein at least some of the first Tele image and the additional Tele images do not overlap each other.
21. The digital camera of claim 18, wherein the configuration to move the FOV.sub.T relative to the FOV.sub.W includes a configuration to move the FOV.sub.T manually.
22. The digital camera of claim 18, wherein the configuration to move the FOV.sub.T relative to the FOV.sub.W includes a configuration to move the FOV.sub.T automatically.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Non-limiting examples of embodiments disclosed herein are described below with reference to figures attached hereto that are listed following this paragraph. Identical structures, elements or parts that appear in more than one figure are generally labeled with a same numeral in all the figures in which they appear. The drawings and descriptions are meant to illuminate and clarify embodiments disclosed herein, and should not be considered limiting in any way.
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
DETAILED DESCRIPTION
(26)
(27)
(28) The operation of a camera such as camera 200 is described in more detail in U.S. patent application Ser. No. 14/455,906. In particular, the prism can perform a tilt (rotation) movement 216 around the X-axis as shown. The rotation motion may be caused by a stepping motor 302 shown schematically in
(29)
(30) The rotation of prism 212 around the X-axis moves the Tele FOV relative to the Wide FOV, causing other portions of scene 106 to become a “Tele image” with higher resolution. Thus,
(31) As mentioned, in
(32)
(33) Note that a similar FOV.sub.T relative positioning effect to that described above may be obtained by in-plane rotating Tele sensor 214 by 90 degrees and by leaving Wide sensor 208 unchanged from its original position in
(34) When a dual-aperture camera described above is included for example in a smartphone, the Tele image (i.e. the part of scene 106 viewed and acquired by the Tele camera) is bound by a frame 932 visible on the smartphone screen.
(35) In still mode, scene 106 is acquired by both cameras, with the Wide camera providing the entire image seen (i.e. the Wide image) and the Tele camera providing the part of scene 106 bound by frame 932. Smartphone 900 further includes, on a front side opposite to the back side, a screen or display 902 displaying a view of scene 106. Screen 902 may display icons or text “A”, “B”, “C”, etc., that provide indications and/or are selectable to perform various operations of the phone and/or the cameras. Such icons or text may be indicative of flash setting, video or stills selection, back or front camera selection, etc. The square boxes surrounding “A”, “B” and “C” are merely illustrative and may have different shape or be removed altogether in some cases. Note that the fact that only three icons are shown is not meant to be limiting, and that more or fewer icons may be displayed and/or selectable at any time during or prior to image acquisition by the cameras and/or during display of acquired images. In an embodiment of the dual-aperture camera as in
(36) In various embodiments and as described in more detail in PCT/IB2016/056060, smartphone 900 may have a user interface that includes a single camera icon (or “button”) 908 and a “two-camera” button 910. The two-camera button may appear on the screen when the FOV of the scene (FOV.sub.scene) is greater or equal to FOV.sub.T. As described in detail in PCT/IB2016/056060, the user interface displays visually the almost exact FOV.sub.T and enables simple acquisition of the image within FOV.sub.T, thereby providing a Tele image with the highest resolution enabled by the Tele camera. The user interface also enables simultaneous acquisition (with a single input through the user interface, i.e. using two-camera button 910) of a Wide image and a Tele image. Image fusion of the Wide and Tele images or video streams can take place on the capturing device or in a cloud environment.
(37) The present inventors have determined that, advantageously, a user interface as described above can be used to “drag” frame 932 (and FOV.sub.T) on screen 902 to bring different parts of the scene into FOV.sub.T. That is, the dragging of frame 932 is translated into rotation of the “folded” path prism, such that the higher resolution Tele image “moves” to different parts of the scene. The dragging may be performed by a firm touch of the screen by a finger 920 and movement of the finger across the screen. In
(38) As described in detail in PCT/IB2016/056060, in terms of image acquisition, a user may press two-camera button 910 to simultaneously acquire two images, the Wide image of scene 106 at its respective (lower) image resolution and the Tele image of region (frame) 932 (or 932′) at its respective (higher) image resolution. The two images may be stored in an on-board storage (such as “camera roll” in an iPhone) and may be displayed or downloaded for further use as known in the art. The user may press single camera button 908 to acquire only the Wide image, which can further be stored, displayed and downloaded for further use. The user may choose for display on screen 902 only the Tele image by, for example, double-tapping or pressing at any point on the screen within frame 932 (or 932′). This action leads to display of the Tele image on the entire screen. The Tele image (only) can then be chosen for acquisition by pressing on single camera button 908. The acquired Tele image can then be stored, displayed and downloaded for further use as above. The two images can also be fused (on the camera hosting device or in a cloud) into a composite image with a portion marked by a frame 932 or 932′ formed by the higher-resolution Tele image, and with a peripheral portion formed by a peripheral portion of the relatively lower resolution Wide image.
(39) Clearly, frame 932′ (and the Tele FOV) may be dragged to any intermediate position (not shown) between the right and left positions. In other words, the Tele FOV may be moved laterally on the screen to a number of partially overlapping or non-overlapping (but touching) positions, from a right-most position (at a right screen edge 940) to a left-most position (at a left screen edge 942) or vice-versa, and an entire image of the scene may be “stitched” together from the partially overlapping or non-overlapping Tele images. For example, as shown in
(40) Note that the direction of prism rotation and the consequent movement of FOV.sub.T relative to FOV.sub.W on a smartphone (or any other host device) screen depends on the geometry of the assembly of the dual-aperture camera in the host device. The description above relates to one particular such geometry. In a different geometry, the prism rotation directions and the resulting FOV.sub.T movement may be in opposite directions to those described above.
(41) The devices, used interface and associated methods disclosed above may be used for automatic movement or “automatic adjustment” of the Tele FOV for e.g. tracking a subject in an autonomous manner We refer to a camera mode that performs automatic Tele FOV movement to track an object or subject of interest as “autonomous Tele FOV tracking”. The autonomous Tele FOV movement is in response to recognition (through e.g. the smart-phone camera) of the object or subject of interest, and the Tele image focuses on and displays the object or subject of interest. The object recognition may be performed using any of the methods known in the art. An example of autonomous Tele FOV tracking is shown in
(42)
(43) While the smartphone shown in
(44) Wide and Tele images and/or video streams can be recorded during the automatic tracking mode and fused together to form a composite image or composite video stream. This fusion can be applied on the camera hosting device or alternatively, Wide and Tele images or video streams can be uploaded to the cloud for applying this fusion operation. Each composite image may also be formed with FOV.sub.W by scanning with the Tele camera, stitching a plurality of Tele images to provide a “stitched” Tele image, then fusing the stitched Tele image with the Wide image. This is advantageous in that the Wide image captures the entire scene simultaneously, while the Tele images to be stitched together are consecutive, so one can overcome motion or occlusions in the scene if required. The stitching of the Tele images and/or the fusion of the stitched Tele image with the Wide image may also be performed in a cloud.
(45) While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. The disclosure is to be understood as not limited by the specific embodiments described herein, but only by the scope of the appended claims.