Roberts' home after 1780 was at 'The Grove', now Grove House, which sports an ancient mulberry on the front lawn