I faced the same problem with my 86 corolla....its really dangerous. your brakes might suddenly fail one day. there might b one of the following problem
1. Rear brake cylinder washers might b torn out.
2. Servo needs replacement.
Just remove wheels of ur car n c whether their is any fluid leakage near drums.u can easily notice oil around Discs or near cylinders.