The area will be turned into a park, and the council says it will include improved links through the town centre.
The work is being funded through the Growth Zone and is part of wider improvements across the town centre.
The site has historic links to the old Walnut Fair which dates back to 1314 and today sees the humble walnut harvested and celebrated.
