A system for detecting and associating elements in an image comprises detecting a plurality of text tokens in a query image. A first element is determined based on an entry point object list and the plurality of text tokens, wherein the entry point object list comprises of text objects or template shapes present in the query image. A plurality of region of interests (ROIs) around the first element in the query image are determined and a plurality of ROI images are created from the query image based on the plurality of ROIs. Determine a potential second element present in the plurality of ROI images. Generate a confidence score for each of the potential second elements. Filter the results based on the confidence score and a predetermined threshold to determine a second element. Subsequently, associate the second element with the first element as single component.