In recent years, ubiquitous and pervasive scenarios have emerged as a complex ecosystem where differentiated software/hardware components interoperate wirelessly and seamlessly. The goal is to enable users to continuously access services and contents, and to always get the best out of their current environment and available resources. In such dynamic and flexible scenarios, the need emerges for flexible and general solutions for continuous runtime self-reconfiguration and self-optimization of ubiquitous support software systems. This paper proposes a fully reconfigurable middleware approach that aims at reconfiguring complex software systems made up of heterogeneous off-the-shelf components from both functional and non-functional perspectives. Our middleware can also extend already existing and non-reconfigurable middleware/applications in an easy and flexible way, with no need to re-design them. The proposed design principles have been practically applied to the implementation of a runtime self-reconfigurable middleware called Off-The-Shelf Ready To Go (OTS-RTG), implemented on top of iPOJO. The reported experimental results both exhibit a limited overhead and show the wide applicability of the proposed solution to many application scenarios, including complex, industrial, Enterprise Service Bus-based ones.