I was used (netscreen) that I could set a tcp connection timeout per service, e.g. custom service tcp/9444 timeout 02:00:00.
But with the FWSM I could only specify a connection timeout for all tcp connections? Is that true or is there a way to specify it only for certain connections?
It depends on which version of code you are running on your FWSM. Roughly speaking
2.x = pix 6.x
3.x = pix 7.x
On pix 6.x you could only set the timeout globally for all tcp connections. With v7.x you can do it on a per port basis. So if you have v3.x on your FWSM yes you can do this.
See attached link for more details