Its perfectly normal to visit khunjerab and then back to gilgit for night stay...IMO don't stay in hunza,,,go to gilgit for night stay...it will be good for your next journey to skardu..it will reduce almost 3 hours...
if you start your journey for khunjerab at 0800 hrs,, you will be there at 1200 hrs easily..stay there for 1 hour..and then start your journey back...make a short stops near passu, and hussaini suspension bridge..you will be in gilgit at 1900 or 2000 hrs easily,,,,