A net 53% of respondents to Alexander's monthly mortgage adviser survey reported a decrease in the number of investors coming forward for mortgage advice.
While the figure marks a slight improvement on previous months, the data point "clearly signals substantial wariness on the part of investors", Alexander said.
A net 78% of respondents noted a decrease in investor activity in April as the Government's housing reforms spooked the market. The latest survey underlines the ongoing uncertainty facing advisers as new regulations are imposed on the sector.
The phased removal of interest deductibility, and extension of the bright-line test, alongside an expected increase in interest rates, are poised to take the heat out of the sector, Alexander said.
While a better understanding of the tax changes might ease the pace of investor withdrawal, a host of factors imply the trend will continue to head in a downward trajectory.
Alexander added: "Discussion about rising interest rates is growing and the Reserve Bank is to be given the ability to use debt to income lending restrictions in future years. This implies a structural easing off of investor demand."
Elsewhere in the monthly survey, a third of respondents said banks had become more willing to lend, down from 36% in May. Alexander said this "does appear to continue a broad upward trend which started back in April".
The study also revealed that borrowers are less likely to pick a one-year fixed rate.
A gross 30% of advisers say that the one-year term is the favourite of borrowers, down from about 70% in February.
According to the survey, three-year terms are just as popular with borrowers, according to 30% of advisers.