DEVICE FOR DISPLAYING IMAGES, AND USE OF A DEVICE OF THIS TYPE

Abstract

Exemplary arrangements relate to a user communication device (1) for displaying images, which has an image display surface (12), speakers (19), and a microphone (11). At least one camera (10, 10a, 10b) is disposed in at least one region of the image display surface. When the device is used, the image display surface of a first device is positioned relative to a first user (14) such that the eyes (16) of the user are directed toward the at least one camera. Subsequently, additional users (20, 21) are positioned in front of additional devices in remote locations. The images of the additional users are displayed in certain regions (4 to 9) of the image display surface of the device of the first user, such that the users all appear to be positioned around a virtual table and able to look directly at one another.

Claims

1.-21. (canceled)

22. Apparatus comprising: a user communication device including: a display surface, wherein the display surface is configured to display images perceivable by a user positioned outward from the display surface, wherein the display surface includes a plurality of horizontally disposed regions, at least one camera, wherein the at least one camera is operative to record data corresponding to a plurality of horizontally disposed views directed outward from each of the regions of the display surface, circuitry, wherein the circuitry is in operative connection with the at least one camera, wherein the circuitry is operative responsive to the data corresponding to the plurality of horizontally disposed views, to determine data corresponding to a respective camera view directed outward from each of the respective regions of the display surface.

23. The apparatus according to claim 22 wherein the display surface is curved such that opposed horizontal sides of the display surface extend outward at an acute angle relative to a central area of the display surface, wherein the at least one camera comprises a plurality of horizontally disposed cameras at respective locations along the curved surface, each with a respective view outward from the respective location along the curved surface.

24. The apparatus according to claim 22 wherein the display surface comprises a spherically curved surface having a facing direction outward toward the user.

25. The apparatus according to claim 22 wherein the display surface is concave and has a radius of greater than 0.5 m.

26. The apparatus according to claim 22 wherein the display surface is concave and has a radius of greater than 0.8 m.

27. The apparatus according to claim 22 wherein the at least one camera comprises a plurality of cameras, wherein each of the plurality of cameras is linearly aligned horizontally with the other cameras, and each camera is horizontally disposed from each of the other cameras.

28. The apparatus according to claim 22 wherein the at least one camera comprises a plurality of cameras, wherein the cameras are disposed adjacent to an upper periphery and a lower periphery of the display surface in a respective region, wherein the circuitry is operative to calculate data corresponding to a virtual camera view from a location in the respective region intermediate of the upper periphery and the lower periphery.

29. The apparatus according to claim 22 wherein the at least one camera comprises a plurality of cameras, wherein each of the plurality of cameras is horizontally disposed from each of the other cameras, is positioned in horizontally linearly aligned relation with each of the other cameras, and has a respective view outward from the display surface.

30. The apparatus according to claim 22 wherein the at least one camera comprises a plurality of cameras, wherein each of the plurality of cameras is horizontally disposed from each of the other cameras, is positioned in a respective region, is positioned in horizontally linearly aligned relation with each of the other cameras, and has a respective view outward from the display surface in the respective region.

31. The apparatus according to claim 22 wherein the display surface is concavely curved, wherein the at least one camera comprises a plurality of cameras, wherein each of the plurality of cameras is horizontally disposed from each of the other cameras, is positioned in a respective region, is positioned in horizontally aligned relation with each of the other cameras, and has a respective view outward from the concavely curved display surface in the respective region.

32. The apparatus according to claim 22 wherein the at least one camera comprises a plurality of cameras, wherein each of the plurality of cameras is horizontally disposed from each of the other cameras, is positioned in a respective region, is positioned in horizontally aligned relation with each of the other cameras, is vertically positioned at an eye level of the user, and has a respective view outward from the display surface in the respective region.

33. The apparatus according to claim 22 wherein the circuitry is operative to output signals corresponding to the data corresponding to each of the camera views.

34. The apparatus according to claim 22 wherein the circuitry is operative to output signals corresponding to the data corresponding to each of the camera views, wherein the signals are usable to generate output images on a respective further display of at least one further user communication device disposed remotely from the user communication device.

35. The apparatus according to claim 22 wherein the device further includes a microphone, wherein the microphone is configured to record audible sounds from the device user, wherein the microphone is in operative connection with the circuitry, wherein the circuitry is operative to output signals corresponding to the data corresponding to each of the camera views and the recorded audible sounds.

36. The apparatus according to claim 22 wherein the display surface comprises a reflection surface configured to display images projected onto the reflection surface by at least one projector.

37. The apparatus according to claim 22 wherein the device further includes a plurality of speakers, wherein the speakers are horizontally disposed from one another.

38. The apparatus according to claim 22 wherein the device further includes a plurality of speakers, wherein the plurality of speakers are configured to produce sounds that are user perceivable as originating from only a selected one of the regions.

39. The apparatus according to claim 22 wherein the device further includes a plurality of disposed speakers, wherein each of the plurality of speakers is horizontally disposed from each of the other speakers, and horizontally linearly aligned with each of the other speakers.

40. The apparatus according to claim 22 wherein the circuitry is operative to receive signals produced by another user communication device that is remotely located from the user communication device, responsive to the received signals, to cause images corresponding to data corresponding to at least one view of at least one camera of the another user communication device, to be displayed in one respective region of the display surface.

41. The apparatus according to claim 22 wherein the device further includes at least one speaker, wherein the circuitry is operative to receive signals produced by another user communication device that is remotely located from the user communication device, responsive to the received signals, to cause images corresponding to data corresponding to at least one view of at least one camera of the another user communication device, to be displayed in one respective region of the display surface, and to cause the speakers to produce sound that is user perceivable to originate from the one respective region of the display surface.

42. The apparatus according to claim 22 wherein the circuitry is operative to receive signals produced by each of a plurality of other user communication devices, each of which other user communication devices is remotely located from the user communication device, responsive to the received signals, to cause respective images to be displayed in different respective regions of the display surface, wherein the images displayed in each respective region correspond to data corresponding to at least one view of at least one camera of one other respective user communication device.

43. The apparatus according to claim 22 wherein the circuitry is operative to receive signals produced by each of a plurality of other user communication devices, each of which other user communication devices is remotely located from the user communication device, responsive to the received signals, to cause respective images to be displayed in different respective regions of the display surface, wherein the images displayed in each respective region correspond to data corresponding to at least one view of at least one camera of one other respective user communication device, wherein the images in each respective region are horizontally aligned on the display surface.

44. The apparatus according to claim 22 wherein the circuitry is operative to receive signals produced by each of a plurality of other user communication devices, each of which other user communication devices is remotely located from the user communication device, responsive to the received signals, to cause respective images to be displayed in different respective regions of the display surface, wherein the images displayed in each respective region correspond to data corresponding to at least one view of at least one camera of one other respective user communication device, wherein the images in each respective region are horizontally aligned on the display surface and at user eye level.

45. The apparatus according to claim 22 wherein the circuitry is operative to receive signals produced by each of a plurality of other user communication devices, each of which other user communication devices is remotely located from the user communication device, responsive to the received signals, to cause respective images to be displayed in different respective regions of the display surface, wherein the images displayed in each respective region correspond to data corresponding to at least one view of at least one camera of one other respective user communication device, wherein the respective images in each respective region are horizontally aligned on the display surface and the images in each respective region correspond to respective user positions around a common virtual table with the user.

46. The apparatus according to claim 22 wherein the device further includes at least one microphone, and at least one speaker, wherein the circuitry is operative to receive signals from each of a plurality of other user communication devices, each of which other user communication devices is remotely located from the user communication device, responsive to the received signals, to cause respective images to be displayed in different respective regions of the display surface, wherein the images displayed in each respective region correspond to data corresponding to at least one view of at least one camera of one other respective user communication device, wherein the respective images in each respective region are horizontally aligned on the display surface, and the images in each respective region correspond to respective positions around a common virtual table with the user, the speakers to provide audible outputs, wherein audible outputs corresponding to signals produced by each other respective user communication device are provided by the at least one speaker to be user perceivable as coming from the respective region in which the images corresponding to the other respective user communication device are displayed.

47. The apparatus according to claim 22 wherein the circuitry is operative to determine a facing direction of the user, wherein the circuitry is operative to cause the at least some of the images displayed to the determined responsive at least in part to the facing direction.

48. The apparatus according to claim 22 wherein the circuitry is operative to receive signals produced by each of a plurality of other user communication devices, each of which other user communication devices is remotely located from the user communication device, responsive to the received signals, to cause respective images to be displayed in different respective regions of the display surface, wherein the images displayed in each respective region correspond to data corresponding to at least one view of at least one camera of one other respective user communication device, wherein the at least one view corresponds to a determined facing direction of a remote user at the other respective user communication device, wherein the respective images in each respective region are horizontally aligned on the display surface and the images in each respective region correspond to respective user positions around a common virtual table with the user.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0034] FIG. 1 shows a perspective diagram view of an exemplary user communication device and a first user in front of an image display surface,

[0035] FIG. 2 shows a cross section through the arrangement shown in FIG. 1.

[0036] FIG. 3 shows the exemplary device used in a virtual video conference table arrangement with two discussion partners.

[0037] FIG. 4 shows the exemplary device used in a virtual video conference table arrangement with three discussion partners.

[0038] FIG. 5 shows the exemplary device used in a virtual video conference table arrangement with four discussion partners.

[0039] FIG. 6 shows the exemplary device used in a virtual video conference table arrangement with five discussion partners.

[0040] FIG. 7 shows the exemplary device used in a virtual video conference table arrangement with six discussion partners.

DETAILED DESCRIPTION

[0041] The exemplary user communication device 1 shown in FIG. 1 has a large curved spherical screen 2 whose inner side forms the image display surface 3. This exemplary image display surface 3 has six horizontally disposed surface regions 4 to 9. Each region of the exemplary arrangement has respectively at least one camera 10 (numbered only as an example) and at least one microphone (11) (numbered only as an example) as assigned. As shown in the drawings, in exemplary arrangements the display surface is curved such that horizontal sides of the display surface extend outward at an acute angle relative to a central area of the display surface. The exemplary inner surface 12 of the screen 2 is spherically curved and the seat 13 of the first user 14 is configured so that the eyes of the user's head 15 look horizontally to the microphones 11 and the cameras 10. In the exemplary arrangement with the concave spherical surface of the image display surface 3, the user's eyes 16 are arranged approximately at the center of the center point of the screen 2. The radius 17 between the eyes 16 of the user 14 and surface 12 of the screen 2 in an exemplary arrangement is 0.9 m or about 1 m. In some exemplary arrangements the radius may be greater than 0.5 m or greater than 0.8 m. In some arrangements the microphones 11 like the cameras 10a and 10b can be arranged at the periphery of the image display surface such as adjacent the upper and lower edges of the image display surface.

[0042] The exemplary cameras 10 are arranged at the eye level of the user 14 and in the exemplary arrangement are horizontally disposed from one another and aligned along a horizontal line 18.

[0043] In the exemplary arrangement the image display surface 3 is a display with a plurality of image sensors which are referred to herein as cameras 10. Alternatively however, in some arrangements the display surface may comprise a reflection surface, and an image can be projected onto the image display surface 3 with at least one projector (not shown).

[0044] Alternatively or in addition, instead of the cameras 10 positioned at user eye level, cameras 10a can also be arranged on the periphery of the display surface such as adjacent an upper edge of the image display surface and cameras 10b can be arranged on the periphery at the lower edge of the image display surface 3. Suitable circuitry such as at least one computer 10c including image determining software is operative to calculate data corresponding to virtual images from the image data corresponding to the views of the cameras 10a and 10b, which calculated data corresponds to the images which would be recorded by cameras 10 positioned within the image display surface at the user eye level.

[0045] In addition to camera 10 and microphone 11, in exemplary arrangements respectively one loudspeaker 19 is arranged in each of the regions 4 to 9 of the image display surface 3. In addition or alternatively the loudspeakers 19 can also be arranged like the cameras 10a and 10b, at the periphery or adjacent at least one edge of the image display surface. Both the microphone recording and the loudspeaker reproduction can be processed through operation of the circuitry as in a stereo effect, so that they enable spatially corresponding recording and reproduction which is matched to the current facing position of the user.

[0046] In practice, two to (n) spaced part user communication devices are used simultaneously for video conference communication of two to (n) users. With increasing number (n) of users, the radius 17 of the screen 2 of each device should be increased. Alternatively to increased size, the depiction regions for presenting images of the discussion participants can be reduced for the same radius 17.

[0047] The devices 1 of the respective users 14 and 20 to 24 are each located preferably at different locations. Communication of signals is carried out between the circuitry of the respective devices.

[0048] When using the user communication devices in an exemplary system, the first user 14 initiates a video conference by means of the circuitry and software using known types of data transmission signals. In this case, the circuitry of the first user device communicates signals which correspond to invitations to the other devices of the users 20 to 24 to participate as discussion partners. In exemplary arrangements the circuitry of the communication device of the first user operates to cause the displayed images of discussion partners to be placed as further users 20 to 24 in order of the receipt of signals received from their devices which signals correspond to agreement to participate. For example, displayed user images may be arranged in the order of agreement to participate in the conference, clockwise at a virtual round discussion table 25. In reality however, each user 14 is located at the geometrical center point of the screen 2 of the respective user communication device 1 used by the participant.

[0049] In an exemplary arrangement, on each image display surface 3 of the respective user communication device of users 14 and 20 to 24, another complete image is displayed. Depending on the number of users 14 and 20 to 24, the image displays are combined from data corresponding to the views of the cameras 10 used by means of the circuitry and software which comprise a computer system for the communication of signals and image and sound processing and transmission.

[0050] During an exemplary video conference, data corresponding to a plurality of images of the first user 14 are recorded using data corresponding to views of a plurality of cameras 10 of the first device 1. The circuitry of the device 1 is operative to determine responsive to the image data, a camera which is arranged closest to the region of the image display surface 3 toward which the viewing (facing) direction 26 of the first user 14 is currently pointing. In addition, the user 20 to 24 corresponding to the region 4 to 9 onto which the viewing direction 26 of the first user 14 is directed, is determined through operation of the circuitry. The circuitry is operative to transmit signals corresponding to the data corresponding to the view of this determined camera. This user 20 to 24 is displayed through operation of the circuitry of their device, the image corresponding to the view of that camera 10 which is arranged closest to the region 20 to 24 of the image display surface 3 of the first user's device onto which the viewing direction 26 of the first user 14 is pointing.

[0051] As an example, a video conference with four participants as represented virtually according to FIG. 5 is described hereinafter. Respectively three discussion participants have images thereof displayed on each image display surface 3 of the user communication device of individual users 14 and 20 to 22. The fourth discussion participant is in each case is the user who is using their own user communication device 1. The circuitry of the device 1 causes images of the three users 20 to 22 to be displayed to the first user 14 next to one another in respective horizontally aligned regions on his image display surface 3, and specifically in regions that correspond to the positions horizontally 45° left of the center, centrally at 0° and 45° right of the center. Of course this region arrangement is exemplary and other region arrangements may be used.

[0052] In the exemplary arrangement the eye level of the users 20 to 22, the images of which are displayed to the user 14, corresponds to the eye level of the first user 14. Each user 14 and 20 to 22 has his image captured by being in the respective views of three cameras 10 which are located in the same positions as the displayed images of the other participants: horizontally 45° left of the center, centrally at 0° and 45° right of the center. In the exemplary arrangement when two users 14 and 20 virtually look at each other, camera angles and projection angles are the same. The immersive impression of a direct conversation is obtained and is further reinforced by the radially curved image display surface 3 of the exemplary user communication devices.

[0053] When a first user 14, 20 to 22 addresses another participant in the virtual conference table environment as represented, the circuitry of the devices operates so that the latter will recognize this by means of the viewing direction 26 and the alignment of the head and the depicted upper body of the first user participant as displayed on the display surface of their device. The course of the discussion can therefore be carried out dynamically around a virtual table in the virtual conference environment which simulates a real table discussion at which all participants are physically present.

[0054] A representation of the simplest virtual conference arrangement is shown in FIG. 3. Here the circuitry in each device operates so that it appears on the display surface of the device of each user, that the users 14 and 20 are sitting directly opposite one another and the camera 10 arranged at 0° is assigned to the users.

[0055] In the exemplary representation of the virtual conference table arrangement shown in FIG. 4 the user 14 can direct his viewing direction 26 to the user 20 or to the user 21. If the user 14 faces the viewing direction 26 toward the image user 20 who is displayed on the image display surface 3 in the left region, the camera provided there is used so that the device of user 20 receives signals which correspond to the data which corresponds to the image view of the user 14 looking into the camera on his device image display surface 3, and this image is output on the display surface of the device of user 20. The device of user 21 to whom the viewing direction 26 is not specifically directed, receives signals corresponding to the image of the right-hand camera which shows the user 14 from the side.

[0056] Accordingly, the devices of the users 20 to 24 shown in the virtual conference table arrangements represented in FIGS. 6 and 7 each receive signals corresponding to data of different images, which are produced based in the views of different cameras. Thus, the users 14 and 20 to 24 in FIG. 7 appear to the users to sit optically opposite one another at an angle of 60°. Five persons have data corresponding to respective images produced as viewed in an outward direction from their respective image display surfaces 3 of the users in each case, and for this purpose cameras are used in positions horizontally 60° and 30° left of the center, centrally at 0° as well as 30° and 60° right of the center in order to provide each user 20 to 24 the displayed image corresponding to the user to be in the correct alignment onto the image of the first user 14.

[0057] In the case of (n) participants the exemplary circuitry operates such that the displayed images of the participants sit optically at an angle α=360°/n opposite one another. N−1 persons have image data output on the display surface of each device. The cameras having the views which are used, are those positioned and used in angular positions divided horizontally uniformly into 180°/n. In arrangements that provide virtual seating arrangements at elliptical or oval virtual tables, the angles are adapted accordingly.

[0058] Since the exemplary devices should be suitable for as many discussion participants as possible, as many cameras as feasible are included as part of the user communication device which can then be used for different numbers of discussion participants.

[0059] So that the users 14 and 20 to 24 also get a feeling for which user is speaking directly, the sound of the first user 14 is captured and recorded with a microphone 11 of the first user's device, and in the case of the further users 20 to 24, is reproduced via a loudspeaker which is arranged closest to the region 4 to 9 of the image display surface 3 which displays the image of the participant, toward which the viewing (facing) direction 26 of the first user 14 corresponds when the first user is speaking. Of course it should be understood that this approach is exemplary and in other arrangements the circuitry may be operative to cause outputs from speakers adjacent to the displayed image of the participant who is speaking or from other speakers.

[0060] The exemplary devices enable communication by virtually providing conferences with participants around a table by employing the eye-to-eye principle of matching viewing directions when recording and reproducing the images of the users and preferably also simulating sound of an in-person meeting.

[0061] In some arrangements the screen 2 of the device 1 can, for example, be configured as a flat shell or be mounted on a foldable or displaceable substructure. At eye height of the first user, small cameras can then be integrated in different angular positions in the image display surface. With a diameter of, for example, about 3 mm for a camera, at the viewing distance of 0.9 m for example, this defect in the display caused by the presence of the cameras is barely perceptible.

[0062] When projection onto the display surface takes place by means of a projector, the projection can be deflected via a spherical mirror. In the case of larger radii a plurality of projectors can be used. The projector or projectors may be positioned above the screen. An undistorted image display is achieved by means of shifting, warping and/or projection mapping. The deflection can also be accomplished, for example, by means of a special lens such as a fish-eye lens.

[0063] Thus the exemplary arrangements achieve improved operation, eliminate difficulties encountered in the use of prior devices, systems and methods, and achieve the useful results described herein.

[0064] In the foregoing description, certain terms have been used for brevity, clarity and understanding. However no unnecessary limitations are to be implied therefrom because such terms are used for descriptive purposes and are intended to be broadly construed. Moreover the descriptions and illustrations herein are by way of examples and the new and useful features and relationships are not limited to the exact features and relationships that have been shown and described.

[0065] Further it should be understood that the features and/or relationships associated with one exemplary arrangement can be combined with features and/or relationships from another exemplary arrangement. That is various features and/or relationships from the various exemplary arrangements can be combined to produce further arrangements. The inventive scope of the described arrangements is not limited solely to the exact arrangements that have been shown or described herein.

[0066] Having described features, discoveries and principles of the exemplary arrangements, the manner in which they are constructed and operated, and the advantages and useful results attained, the new and useful features, devices, elements, arrangements, parts, combinations, systems, equipment, operations, methods, processes and relationships are set forth in the appended claims.