I am fine when people buy the device and use and modify it for their own purpose
It seems to me that you're honouring your obligations under the GPL (and therefore under LGPL). There will of course be the obligations to make source available, etc., but you don't ask about those, so I'm assuming you have them covered.
Can I reject service calls for modified devices?
Many vendors supply devices which incorporate software covered by the GPL and consider the warranty voided if you install your own software on them - just about any company who's ever sold an Android phone fits into this category. If you've ever tried to get warranty support on an Android phone that's been Cyanogenmodded or LineageOSed, this will not be an alien idea. I don't say it's a nice thing to do, but lots of other companies do it without issue. Be clear with your customers about your policy.
Can I prevent modified devices from using the service?
Legally, it's not clear to me. GPL3 s6 requires that you provide the user with "installation information", which
must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.
Whether access to a network service constitutes "continued function" will, I suspect, depend on how useful the device is without that service.
But the point may be moot, because you may find it technically difficult or even impossible to do this; free software has a long history of precisely emulating closed devices in order to obtain service over a network. Consider, for example, get_iplayer; the BBC tries very hard to distinguish between an iphone connecting to view a programme, and a Linux box running get_iplayer doing the same, but has not reliably been able to do so.
Can I remove my proprietary bits as soon as the user is modifying his device (after a stern warning of course)
I wouldn't. Not only is it unlikely to work reliably (see above), but many jurisdictions will take it amiss if you are accessing customer devices to change the contents of them after the customer has pretty clearly expressed a desire to remove the device from your control (by installing modified software). And of course, and as ever, IANAL/IANYL.