The company made the announcement after a 90 day consultation period with workers' union Unite.
The global confectionery company first announced its plans to close the Somerdale plant near Bristol in October last year, citing cheaper production costs in Eastern Europe as the reason for the move.
The company is also planning to reorganise operations at another UK plant in Bourneville, near Birmingham.
"In order to continue to compete in the extremely competitive confectionery industry in the future we need to make the right decisions today," said managing director Trevor Bond at the time.
The decision to go ahead with the closure after the allotted discussion time has led to angry protest over food miles from Unite, in what is one of the first cases of a union using environmental arguments to try and combat the loss of UK food manufacturing facilities.
According to Brian Revell, Unite national organiser for food and agriculture, 98 per cent of the products currently made at the UK Somerdale plant, including Fudge and Crunchie bars, are sold within the same country.
He accused Cadbury of ignoring "powerful environmental arguments by adding food miles to well known products," claiming that once produced in Poland, these chocolate bars will then be transported back to the UK.
But Cadbury spokesperson Tony Bilsborough told ConfectioneryNews.com that Unite had got its figures wrong.
"According to an internal independent audit, there will actually be a net reduction in Cadbury emissions, because of the fact we are closing a site," he said.
"And we are not building any new plants in Poland , but will be working with sister companies which have been established in the country for some time now."
Cadbury claims to take considerable care over the environmental impact of its operations, and established the 'Purple Goes Green' programme in July last year.
As part of the strategy, the company targets a 10 per cent reduction in packaging used per tonne of product, and a 25 per cent cut in the more highly packaged and gift items.
Cadbury also aims to have 60 per cent of its packaging made from biodegradable materials.
Last autumn the company was in fact highlighted by the Climate Disclosure Project as the best in its class for actions taken against climate change.
"Cadbury Schweppes was judged among the best in its sector, and distinguished by the disclosure of its strategic awareness of the risks and opportunities of climate change, as well as the quality and effectiveness of programmes put in place to reduce emissions," the company claimed yesterday.
The decision to go ahead with reorganisation of the company will result in some human damage, however, because, as Cadbury has already stated, about 700 UK workers will lose their jobs.
The loss of jobs will contribute to an 'industrial desert' in the country, Unite said, today calling for a boycott of Cadbury products in the UK.
In October Revell also accused Cadbury of "blatant profiteering", saying that the company is set to make a huge profit from stopping operations on a site that has "high land value".