Well, you've not driven one certainly. I own the green one in this group, and for over 2 years of ownership I have taken it up to Azad Kashmir, done little bit of off road there, but the engine power is enough to pull this little monster, and overtake lots of trucks going uphill, with couple of passengers with me.
As Rameez pointed out, these small 4x4s are very capable, and due to its small size, and light weight, it can go to places where big mighty get some trouble.
In this trip, there was complete off-road rocky track for about 2KMs, both of the Pajero Minis in our group got through without a hitch.
Don't listen to hearsay, either ask from people who own one, or experience it for yourself to make an opinion 