The party lost councillors in places including Tunbridge Wells, Surrey and Oxfordshire, as well as the West of England mayoralty..