Recently, maskless lithography (ML) systems have become popular in digital manufacturing technologies. To achieve high-throughput manufacturing processes, digital micromirror devices (DMD) in ML systems must be driven to their operational limits, often in harsh conditions. We propose an instrument and algorithm to detect DMD malfunctions to ensure perfect mask image transfer to the photoresist in ML systems. DMD malfunctions are caused by either bad DMD pixels or data transfer errors. We detect bad DMD pixels with $20{ imes}20$ pixel by white and black image tests. To analyze data transfer errors at high frame rates, we monitor changes in the frame rate of a target DMD pixel driven by the input data with a set frame rate of up to 28000 frames per second (fps). For our data transfer error detection method, we verified that there are no data transfer errors in the test by confirming the agreement between the input frame rate and the output frame rate within the measurement accuracy of 1 fps.