Hello,
Sorry, I couldn’t reproduce the issue on our server. When I delete any widget from the Customizer and Publish the changes, it’s deleted forever. Anyway, please try deleting them from Appearance > Widgets. On this page, open a widget position, then open a widget and click Delete link at the bottom of the widget’s options panel.
Best regards,
Milos