A lot of relay boxes will both detect the switch position and toggle the power, so you should be able to get rid of all smart plugs that you were planning to use for this light.
The relay should have these connections:
- Live input from breaker box
- Neutral (both from breaker box and to socket)
- Switched live output to socket
- Connection to one side of the manual switch
- Connection to the other side of the manual switch