A feasibility study and the optimal design was conducted for the application of a hybrid motor with HTPB/LOX combination to the first stage of an air launch system. The feasibility analysis showed that the hybrid motor could successfully be used as a substitute for the solid rocket motor of the first stage of the Pegasus XL if the average specific impulse ($I_{sp}$) of the hybrid motor could be approximately 350sec. And the optimal design of the hybrid motor was carried out for a given mission requirement with selected design variables: number of ports, initial oxidizer flux, combustion chamber pressure, and nozzle expansion ratio. The design results show the hybrid motor can be successfully applicable to the air launching vehicle. And, the optimal design can meet physical constraints of length and diameter imposed by the mother plane installation for a 3.5kg nano-sat to the orbit located on the $200{ imes}1500km$ elliptic orbit plane. Also, it was found that the design results would provide nearly the same design configurations regardless of the selection of objective functions: either mass or length of the engine.