In 2006, Zhang and Wang proposed a data hiding scheme based on Exploiting Modification Direction (EMD) to increase data hiding capacity. The major benefit of EMD is providing embedding capacity greater than 1 bit per pixel. Since then, many EMD-type data hiding schemes have been proposed. However, a serious disadvantage common to these approaches is that the embedded data is compromised when the embedding function is disclosed. Our proposed secure data hiding scheme remedies this disclosure shortcoming by employing an additional modulus function. The provided security analysis of our scheme demonstrates that attackers cannot get the secret information from the stegoimage even if the embedding function is made public. Furthermore, our proposed scheme also gives a simple solution to the overflow/underflow problem and maintains high embedding capacity and good stegoimage quality.