A sensor fusion system associated with a vehicle includes a sensor interface communicatively coupled to a plurality of sensors in the vehicle and a vehicle experience system. The sensor interface comprises an input receiving data from each of the plurality of sensors and an output configured to output fused vehicle data based on the data received from the plurality of sensors. The vehicle experience system is coupled to the output of the sensors interface to receive the fused vehicle data. The vehicle experience system includes one or more processors and a non-transitory computer readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to control at least one parameter of the vehicle based on the fused vehicle data.

A vehicle arrangement includes a display device for displaying at least one item of information on a display surface for at least one person being transported by the vehicle, a detection device for detecting disturbance of the view of the person of the display surface, and a further component which is compensating the detected disturbance of the view of the person of the display surface at least partially by at least one of movement relative to the display surface and alteration of its external shape. A vehicle includes at least one such vehicle arrangement, and a method provides a display in or on a vehicle.

A method and a system for enhancing a user experience of an occupant in a vehicle that includes a step of detecting a change of the occupant's line of gaze and a step of outputting a notification, if a change of the occupant's line of gaze is detected or if a predetermined time period has elapsed. The notification is provided on an output device based on the detected occupant's line of gaze.

A vehicle display system for a vehicle, the vehicle display system including: a display device including at least one display; at least one processor; and at least one computer memory that is operably connectable to the at least one processor and that has stored thereon instructions which, when executed, cause the at least one processor to perform operations including: acquiring sitting position data of a user in the vehicle; and providing a control signal to adjust a viewing angle of the at least one display in accordance with the sitting position data of the user.

The present invention includes: a visual perception target determination unit for determining a visual perception target by using a saliency model for determining that an object to be visually perceived at a glance is a visual perception target, a surprise model for determining that an object behaving abnormally is a visual perception target, and a normative model for determining that an object to be visually perceived by a viewing action of a driver serving as a norm is a visual perception target; and a visual guidance unit for determining whether or not an overlooked visual perception to which a line-of-sight direction detected by a line-of-sight detection unit is not directed is present, and guiding a line-of-sight of a driver toward the overlooked visual perception target, when the overlooked visual perception target is present.

Some embodiments described herein relate to a multimodal user interface for use in an automobile. The multimodal user interface may display information on a windshield of the automobile, such as by projecting information on the windshield, and may accept input from a user via multiple modalities, which may include a speech interface as well as other interfaces. The other interfaces may include interfaces allowing a user to provide geometric input by indicating an angle. In some embodiments, a user may define a task to be performed using multiple different input modalities. For example, the user may provide via the speech interface speech input describing a task that the user is requesting be performed, and may provide via one or more other interfaces geometric parameters regarding the task. The multimodal user interface may determine the task and the geometric parameters from the inputs.

A terminal is provided to recognize a touch input and a gesture input intended by a user. A vehicle includes the terminal configured to display buttons to be selected by a user, and receive a touch input as user input. An image input receives an image of the user for receiving a gesture input as the user input and a controller divides an area of the terminal into a first area and a second area. The controller determines a button selected by the user among buttons displayed in the first area based on the touch signal output by the touch input, and determines a button selected by the user among buttons displayed in the second area based on a finger image and an eye image in the image.

A camera system is provided. The camera system includes an image sensor, at least one light source, and a processing unit. The image sensor is configured to capture a plurality of images. The processing unit is configured to perform the following instructions. A plurality of reflection values on at least one subject in the captured images is acquired. A relationship between a luminance level of the light sources and a reflection level on the at least one subject is obtained. A luminance configuration is determined according to the relationship between the luminance level of the light sources and the reflection level on the at least one subject. A luminous power of at least one of the light sources is adjusted according to the luminance configuration.

Vehicle systems and methods for determining a target position are disclosed. A vehicle includes a user detection system configured to output a gesture signal in response to a hand of a user performing at least one gesture to indicate a final target position. The vehicle also includes a user gaze monitoring system configured to output an eye location signal that indicates an actual eye position of the user. The vehicle also includes one or more processors and one or more non-transitory memory modules communicatively coupled to the processors. The processors store machine-readable instructions that, when executed, cause the one or more processors to determine a first point and a second point located on the hand of the user based at least in part on the gesture signal from the user detection system. The first point and the second point define a pointing axis of the hand of the user.

An ECU as a vehicle device includes an image generation unit that generates an image in a predetermined reference coordinate system, a viewpoint position identification unit that identifies a viewpoint position of a driver; a superimposed position identification unit that identifies a display position at which the image is displayed within an eyesight of the driver; an image conversion unit that generates a converted image obtained by converting the image into a coordinate system having the viewpoint position of the driver as a reference point; and a notification unit that displays a display image falling within the converted image so as to be superimposed on the eyesight of the driver with the projection device.