Pretty much a waste of time and money.
To be any good at all - and this can apply to cheap speakers as much as expensive one - the speaker needs to be designed as a whole. Integration between the drivers, the enclosure, expected placement, with the crossover design and other aspects is a colplete design. There is no simple way of swapping out a single portion of the design.
Determining what you mean by high-end for a driver isn't trivial either. Lots of expensive drivers are really only high end because of their price, and not because of their price performance.
A well thought out design with quite cheap drivers (and there are lots of these pouring out of China now) can be made to work well. A skilled designer can work with the flaws and limitations of the drivers to produce something better than the sum of the parts. Of course an uncaring designer will produce something mediocre. If you have a system from a good designer you can be assured of only making it worse by messing with it. If it is from a poor designer, you can be assured that unless you have good design skills and experience, you won't be able to perform the integration of a new driver into the design anyway.