Too much traveling....why don't you plan to cover one side this time...and plan for swat at another time.....I had 2 trips to Kashmir with my family in May and June ... border situation is under control....
With 2 small kids, it will be too hectic...and they will get bored as well