A couple of days ago I was away and my Android 6 phone did not get any non-wifi internet connection anymore. All I saw was an exclamation mark beside the cellular connection icons:
Enabling or disabling the data connection did not help, nor did rebooting change anything. Only when WiFi was enabled and connected, the exclamation marks went away.
Solution
The exclamation marks appear when there is no internet connection - or rather the phone cannot ping some google server.
In the end I found a reddit thread comment that gave me the hint: Reset the SIM card's APN settings. On Android 6 I did that as follows (German):