If you mean this button:

...you can just remove that by clicking the "Remove from Toolbar" in the right-click menu (as shown).
If you mean this button:

...you can just remove that by clicking the "Remove from Toolbar" in the right-click menu (as shown).