Just some random thoughts...
You have to bolt the spacer lugs onto the wheel hubs. Once you bolt the rim ONTO the spacer, you have NO way of detecting how tight the spacer lugs ARE. Loose spacers can have catastrophic consequences.
Poorly constructed spacers may well be prone to corrosion or buckling or cracking. This can be problematic, since these spacers can't be visually inspected without removing the wheel. Again, spacer failure may have serious consequences.
Unless the spacers are VERY tightly torqued onto the hub lugs, there may be some play left over...which may increase wear and tear to the hubs, axles and suspension.
Best way to get around this (in a general manner of speaking), is to ensure your spacers are well constructed and fit well, from the start. Keep checking them regularly for fit and tightness.
And it should be realised, that in some cases, spacers are the ONLY way to go, without resorting to drastic (and costly) changes. Your case, Imran, seems pretty much one such case, where you NOT using spacers, may lead to catastrophic tyre failure from rubbing against the suspension. I guess all things should be considered, according to the situation they merit.