Please find attached diagram of existing configuration.
As is now, RouterB and RouterC are not configured for OSPF, only BGP.
If I put a 3750-layer3-new switch and make the switch do the intervlan routing (instead of RouterB), do I necessarily need to configure OSPF in that switch? I plan to configure iBGP on the switch and I just want to double check whether the switch could route accordingly without OSPF or another IGP.
You really dont need OSPF in this scenario. This is a very small topology and you can simply disable iBGP synchronization on 3750, to have IBGP routes advertised between the peers. If you want to can also make a full-mesh BGP to have all the routes advertised to iBGP peer. You need an IGP typically in a very big bgp network.
You have to make sure that you are not getting the full BGP internet table, 3750 will not be able to support that.
HTH,Please rate if it does.