I too felt such problems with my XLI, but I then changed its tyres to Dunlop 195/65/16, I bought them for Rs. 18,000. Yes I know its increases the height of the car, but I like it higher, also because I do not travel cross-cities too much, and even when I do, I keep my speed under 100KM/H.
I think, Corolla-XLI is a light weight, 4 door family sedan and should be driven as such. I you really want to frequently drive it over 100KM/Hr, do not used 185, use at least 195/55 or 60/14.
The earlier 2 models of Corolla that I kept did not have these problems. I think the cause of the problem is higher in height and more ground clearance.
I also feel that this model of Corolla is inflated like a balloon, it has more mass than comparative amount of required weight, and that may be a contributing factor.
Also, you may also feel the difference if you let the air pass through the windows and just to test, keep your window glasses lowered while overtaking heavy vehicles on higher speeds.
Hope this helps!