At a press conference following the RBNZ's latest Financial Stability Report, Orr was asked whether loan rates could fall further, following action taken by the central bank to push down rates since the Covid-crisis.
Orr acknowledged a "delay working through the system" in passing on cuts since the official cash rate was lowered. He said the Reserve Bank believed banks had "more room to go" to push prices even lower.
Aside from cutting the official cash rate to 0.25%, the Reserve Bank has ramped up the pressure on banks to pass on rate cuts in recent weeks.
Mortgage rates have dropped to record lows in the past week, with Kiwibank offering a 2.65% one-year rate, and ASB offering 2.69% over two years.
It is hoped lower rates will help borrowers survive through the severe recession caused by Covid-19.
The Reserve Bank also spoke about the possibility of extending loan payment deferrals beyond existing six month periods. The RBNZ currently gives banks an exemption so deferrals do not count against their balance sheets.
Orr said any call to extend loan deferrals would "depend on what government policy is at the time, and we would facilitate as best we can what that policy situation is".
Deputy governor Geoff Bascand played down the chances of extending loan deferrals for long periods, pointing out banks "can't do that forever".
"There is a limit before banks would be able to regard those as non-performing," Bascand added. "We would have to work through that at some stage and work out what the approach is."