A system and method is disclosed for guiding invasive medical devices relative to anatomical structures. An imaging device can generate one or more images of the invasive medical device within the patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device with at least one of orientation and position information. An imaging computer system can apply the trained model to unannotated images of the invasive medical device within the patient to determine at least one of a current orientation and a current position of the invasive medical device relative to the one or more anatomical structures within the patient. The images of the invasive medical device, visual orientation information representing the current orientation, and visual position information representing the current position of the invasive medical device relative to the anatomical structures within the patient can be outputted to a display.

A system and method is disclosed for augmenting image data of an invasive medical device having non-medical structures. An invasive medical device having markers with distinct physical shapes relative to other portions of the invasive medical device can be inserted in a patient. An imaging device can generate images of the invasive medical device within the patient. A trained model for the invasive medical device can be trained on annotated images of the invasive medical device annotated with marker and spatial information. An imaging computer system can apply the trained model to images of the invasive medical device having depictions of the markers to determine current spatial information. The depictions of the markers can be identified and correlated with the spatial information from the annotated images. The images and visual spatial information representing the spatial information of the invasive medical device can be outputted to a display.

A system and method is disclosed for automatically determining position information for an invasive medical device based on imaging data. An imaging device can generate 2D images of the invasive medical device within the patient from a vantage point relative to the patient. A trained model for the invasive medical device can be trained on annotated 2D images of the invasive medical device with position information. An imaging computer system can apply the trained model to unannotated 2D images of the invasive medical device within the patient to determine a current position of the invasive medical device. The 2D images of the invasive medical device and visual position information representing the current position of the invasive medical device can be outputted to a display.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for classifying a patient. In one aspect, a method comprises: receiving multi-modal data characterizing a patient, wherein the multi-modal data comprises a respective feature representation for each of a plurality of modalities; processing the multi-modal data characterizing the patient using an encoder neural network to generate an embedding of the multi-modal data characterizing the patient; determining a respective classification score for each patient category in a set of patient categories based on the embedding of the multi-modal data characterizing the patient; and classifying the patient as being included in a corresponding patient category from the set of patient categories based on the classification scores.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for jointly training an encoder neural network and a decoder neural network. In one aspect, a method comprises: updating current values of a set of encoder parameters and current values of a set of decoder parameters using gradients of a reconstruction loss function that measures an error in a reconstruction of multi-modal data from a training example, wherein: the reconstruction loss function comprises a plurality of scaling factors that each scale a respective term in the reconstruction loss function that measures an error in the reconstruction of a corresponding proper subset of feature dimensions of the multi-modal data from the training example.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating multi-modal data archetypes. In one aspect, a method comprises obtaining a plurality of training examples, wherein each training example corresponds to a respective patient and includes multi-modal data, having a plurality of feature dimensions, that characterizes the patient; jointly training an encoder neural network and a decoder neural network on the plurality of training examples; and generating a plurality of multi-modal data archetypes that each correspond to a respective dimension of a latent space, comprising, for each multi-modal data archetype: processing a predefined embedding that represents the corresponding dimension of the latent space using the decoder neural network to generate multi-modal data, having the plurality of feature dimensions, that defines the multi-modal data archetype.

The present disclosure provides a method for treating clinical or pre-clinical skin damage in a skin field of a subject, wherein the skin field has been allocated a skin cancerization field index (SCFI) score of at least 1 as determined by a process comprising the steps of: (i) assessing the number of keratoses in the skin field; (ii) assessing the thickness of the thickest keratosis in the skin field; and (iii) assessing the proportion of the field affected by clinical or subclinical skin damage. Based on the assessments made in (i), (ii) and (iii) the subject is optionally treated by at least one of (a) freezing one or more lesions, (b) shaving, curetting or surgically removing one or more lesions, (c) applying a topical treatment for actinic keratosis, basal cell carcinoma or squamous cell carcinoma, and (d) radiation therapy.

An X-ray imaging system with multiple pulsed X-ray source tubes in motion to perform highly efficient and ultrafast 3D radiography is presented. There are multiple X-ray tubes from pulsed sources mounted on a structure in motion to form an array of X-ray tubes. The tubes move simultaneously relative to an object on a pre-defined arc track at a constant speed as a group. Each individual X-ray tube in each individual source can also move rapidly around its static position in a small distance. When a tube has a speed that is equal to group speed but with opposite moving direction, the tube and X-ray flat panel detector are activated through an external exposure control unit so that the tube stay momentarily standstill. It results in much reduced travel distance for each X-ray source tube and much lighter load for motion system. 3D X-ray scan can cover much wider sweeping angle in much shorter time and image analysis can also be done in real time.

A method of chronologically imaging progress of a patient's dental treatment includes providing an executable application to a portable electronic device, the executable application causing a processor to instruct a user to position an image capture device lens towards the user's face, assess ambient lighting condition(s), superimpose an alignment guide on a display screen, instruct the user to align the alignment guide with the user's upper and lower vermillion borders, and capture an image of the user's teeth. A non-transitory computer readable medium and a system to implement the method are also disclosed.

A method comprises providing a pre-treatment image of a target subject to at least one deep learning model uniquely trained to predict immunotherapy treatment responses. The method further comprises generating, by a processing device, a predicted treatment response score to a treatment based on the single pre-treatment image and the at least one deep learning model. The method further comprises providing, based on the predicted treatment response score, a recommended treatment plan.