Programmable FIR filters are used in various signal processing tasks in medical ultrasound imaging, which are one of the major factors increasing hardware complexity. A widely used method to reduce the hardware complexity of a programmable FIR filter is to encode the filter coefficients in the canonic signed digit (CSD) format to minimize the number of nonzero digits (NZD) so that the multipliers for each filter coefficients can be replaced with fixed shifters and programmable multiplexers (PM). In this paper, a new structure for programmable FIR filters with a improved frequency response and a reduced hardware complexity compared to the conventional shift-and-add architecture using PM is proposed for implementing a very small portable ultrasound scanner. The CSD codes are optimized such that there exists at least one common nonzero digit between neighboring coefficients. Such common digits are then implemented with the same shifters. For comparison, synthesisable VHDL models for programmable FIR filters are developed based on the proposed and the conventional architectures. When these filters have the same hardware complexity, pass-band ana stop-band ripples of the proposed filter are lower than those of the conventional filter by about $0.01{sim}0.19dB$ and by about $5{sim}10dB$, respectively. For the same filter performance, the hardware complexity of the proposed architecture is reduced by more than 20% compare to the conventional SaA architecture.