Iain Gray, Scotland's minister for enterprise, transport and lifelong learning, says there are still severe concerns about further education's financial health.
Mr Gray warned the Association of Scottish Colleges' annual conference that urgent work was needed to clear the backlog of deficits - 22 of the 43 colleges had losses totalling £30 million.
While he was satisfied that significant progress was forecast on eradicating deficits, it was up to board members and principals to ensure the improvements were achieved, he said.
After a mismanagement scandal at Moray College, the committee raised the prospect of allowing the Scottish Further Education Funding Council to intervene in cases of poor governance.
Mr Gray is consulting on increasing the SFEFC's powers, letting it appoint some board members and attend board meetings, and extending ministers' powers to replace governors.
He told the colleges that his review of governance, recommended by the Scottish Parliament's audit committee, was not intended to indicate "any widespread ministerial concern" over leadership.
The sector should expect scrutiny, given the scale of the Scottish Executive's investment in the sector - a 50 per cent cash-terms increase over the past three years - he said. Although this year has seen an increase of only 1.5 per cent, and Mr Gray held out little hope of extra cash.
He announced a one-off £10 million fund to be spent this year on capital equipment, enabling students to gain technical skills needed in the workplace.
Tom Kelly, chief officer of the ASC, said: "We are uneasy about the assumption that more powers to intervene would necessarily avoid mischief. We think they could get us into difficulties of who is in control."