I've had situations where I've had to increase the timeout conn setting to 4hrs or even 8hrs to keep telnet sessions from being dropped during the day.
The main issue is the possibility of something happening that opens a ton of sessions without correctly closing them. That could lead to processor/memory usage, and eventual crash.... Although I've never seen it.