I've the Swift and my brother in law has the City.
I have never driven a Vitz so won't be talking about that.
The City has more power (we raced on the motorway when going to Islamabad and the City very slowly drew ahead). The drive is pretty much the same.
The City has more acceleration as well. The Swift accelerates well but not as fast the City.
The Swift has much much better handling. It corners extremely well and because of that, it's a dream to drive it in the mountains. Honda Civic Reborn is a low car and it corners superbly (have driven that in the mountains) but the Swift's drive in the mountains is much better.
The build quality of the City is better I think. My left window is making noises as it goes up and down, due to dried up water in it's parts. There's too much play in the dashboard plastic.
But all in all, it's also cheaper by 2 lakhs. So I think for the price it's being sold for there's no better option.