I think this is normal, As per the guide there is Enhancement for this behavior if in certain firmware.

Think of phone as three port Switch ( One Port Connected to PC, One Port Connected to Switch, and One port to phone itself).

From the guide

http://www.cisco.com/c/en/us/products/collateral/ios-nx-os-software/identity-based-networking-services/config_guide_c17-605524.pdf

If the device unplugs from behind the phone, the switch cannot rely on link state to know when to clear the session. Dangling sessions can lead to security violations and security holes. Take the situation illustrated in Figure 6 below

Cisco Discovery Protocol (CDP) Enhancement for Second Port Disconnect:. This feature allows a Cisco IP phone to send a CDP message to the switch when a host unplugs from behind the phone. The switch is then able to clear any authenticated session for the indirectly connected host, exactly the same as if the host had been directly connected and the switch had detected a link down event. 

What is firmware and phone model you are using, try to upgrade the same