I wish you had a scanner, as it comes very handy in diagnosing and solving such issues.
The problem apparently looks as if vacuum is leaking inside the IM, you may need to check the following;
a. Air leakage
b. MAP sensor
c. Wiring
I once had a long staning issue similar to yours, found it to be due to wrong gasket placed between the TB and iM. All D series TB gaskets look apparently same but actually they are not, air passages and cavities are not always in the same location on the mating surface of TB.
The issue then surfaced again after sometime and after long efforts this time found it to be due to the female crimp pins inside the ECU connector. Pins designated for MAP sensor signals were not making firm connection with opposite side (male) pins.
Reason why these crimp pins get loose is due to the fact that ECU plugs sit deep inside the receptacle connector and when removing the same you intentionally or unintentionally pull them from the wires.