Actually, the 4.1 ratio diffs in the diesel LC80 are better suited for the 1UZ than the 3.7 ratio diffs of the FZ version. It will make up for the comparative lack of off idle torque. If you choose to use an auto tranny behind the 1UZ then even the 3.7 diffs will be fine. Remember, the 1UZ revs effortlessly so you don't run out of RPMs if you use a 4.1 ratio diff.
The 3400cc 5VZFE is an anemic engine. It makes less power than even a 3.0 L 2JZGE out a Crown. Its not much to write about in a Surf, so I can imagine how dismal it must be in a LC80.
All in all, I would advise sticking with a 1HDFT. It's a marvelous engine. You can start with simply increasing the boost on the stock turbo, instead of upgrading the turbo right out of the gate. Google will give you plenty of results for easy 1HDFT modifications. Plus. there's nothing quite like driving a high performance diesel. No petrol engine can deliver the same kind of torque