An image reading device includes a contact glass, an automatic document feeder, a first reading module, a second reading module, a white reference member, a swinging mechanism, one motor, and a control portion. The swinging mechanism swings the second reading module or white reference member in a direction orthogonal to the conveyance direction. The control portion, by rotating a motor forward, drives a conveying member to convey a document and, by rotating the motor reversely, drives the swinging mechanism with the conveying member at rest to swing the second reading module or white reference member and meanwhile acquire white reference data. When performing a duplex reading process successively for a plurality of sheets of a conveyed document, the control portion acquires the white reference data with the document at rest closely upstream of the second reading module in the conveyance direction.