On a 4507 (IOS 12.2), I believe you need one physical port to be up in VLAN 5 for the VLAN Interface 5 to be up/up.
Is there any way to over-ride that default?
Vlan 5 as management network address and (let's say) Vlan 100 for the user Vlan.
If you are connecting from another physical segment (another switch, for instance) over a inter-switch link - it's the inter-switch link configured as trunk ? If so, Vlan 5 will be in forwarding state and the SVI will be up/up.
If the inter-switch link was to be configured as access-vlan 100, you couldn't connect to the management IP on Vlan 5 even if the SVI was to be up/up since the inter-switch link would contain Vlan5 tag packets - hence the need for the trunk.
I can only see an issue if you were to connect local to the switch and the switch didn't have any other connections active on Vlan 5 (no inter-switch links). On this scenario, implement loopback interfaces.