You can restrict Caddy access to use your tailscale. For instance in your Caddyfile:

For tailscale ip range:

myverycoolserver.duckdns.org {
    @allowed {
        remote_ip 100.64.0.0/10  # Allow Tailscale IP range
    }
    respond @allowed 200  # Allow access
    respond 403  # Deny access for others
    reverse_proxy localhost:YOUR_SERVICE_PORT  # Your service configuration
}

For specific tailscale IP:

myverycoolserver.duckdns.org {
    @allowed {
        remote_ip YOUR_TAILSCALE_IP  # Replace with the specific Tailscale IP
    }
    respond @allowed 200  # Allow access
    respond 403  # Deny access for others
    reverse_proxy localhost:YOUR_SERVICE_PORT  # Your service configuration
}