Isolate the smart TV in restricted VLAN in your home network that can access your local media server but doesn’t allow internet access.

Segmenting a home network like this is also a good idea for smart home/IoT devices.