UTU makes either an HTTP or HTTPS connection to the LMS server on the standard port (i.e. 1741 or 443). So if you connect to your server on 1741 in HTTP mode, you must use that mode for UTU (same for HTTPS).
If you are using HTTP, it is much easier to troubleshoot. Get a sniffer trace of the traffic between the UTU client and the LMS server when making a query. Also, check the Cmapps.log on the server for any errors when you try and run a UTU query.