.NET TcpClient: "An established connection was aborted by the software in your host machine"

My client app was getting the exception suddenly.  After a lot of hair pulling it seems that a minor change in the hostname given to the TcpClient was the culprit.  The client was connecting to a server on the local machine and got the error when using the fully qualified name (eg. machine.domain.local) of the local machine as the hostname for TcpClient.  The problem goes away when I change the hostname parameter to '127.0.0.1'.  It seems like TcpClient sockets has collision issues when connecting to localhost by the network name.