A honing device includes a honing tool, a rotating main shaft to which the honing tool is fixed, an axial direction driver to cause the rotating main shaft to move in the Z-axis direction, a workpiece holder to hold a workpiece while a central axis of a processed hole of the workpiece and the a central axis of the rotating main shaft coincide with each other, and a holder drive unit to, by exerting a driving force on a portion of the workpiece holder corresponding to a point on an extension line of the central axis of the rotating main shaft in a direction along the central axis, cause the workpiece holder to move in a direction along the central axis.