Disclosed are an image processing device, an imaging device, an image processing method, and an image processing program capable of, when recovering a deteriorated image due to a point spread function of an optical system, suppressing the occurrence of artifact and color gradation and achieving reduction in computational costs. The image processing device includes a frequency recovery processing unit which subjects image data acquired from an imaging element by capturing an object image using an optical system to frequency recovery processing using a frequency recovery filter based on a point spread function of the optical system, a gradation correction processing unit which subjects image data subjected to the frequency recovery processing to nonlinear gradation correction, and a phase recovery processing unit which subjects image data subjected to the gradation correction to phase recovery processing using a phase recovery filter based on the point spread function of the optical system.