All hotels are cheap when demand is low.
A hotel that charges 4,000-5,000 in season will charge 1,000 off-season.
A hotel that charges 7,000-9,000 in season will charge 2,500 off-season.
The local cars/jeeps and buses similarly reduce prices. And you get more options, so you can bargain.
That means plan for a vacation between September to April. In between this time you will see rush on holidays and weekends.
Although people from Punjab usually go to Swat on longer holidays (3 days or more). The people from nearby KP cities visit on weekends and also on Eid. But this rush is much less than the rush during summer vacations.
I would suggest you take 3 days off from work and include the weekend. Which means you can leave on a Tuesday night from Lahore and reach Swat on Wednesday morning. Come back from Swat on Sunday.