One of the major goals suggested in distributed system is to improve the performance of the system through the load balancing of whole system. Load balancing among systems improves the rate of processor utilization and reduces the turnaround time of system. In this paper, we design the rule of decision-making and information interchange based on knowledge based mechanism which makes optimal load balancing by sharing the future load state information inferred from past and present information of each nodes. The result of performance evaluation shows that utilization of processors is balanced, the processing time is improved and reliability and availability of systems are enhanced. The proposed mechanism in this paper can be utilized in the design of load balancing algorithm in distributed operating systems.