Ping Traffic is considered to be the least significant by the switch CPU. If you aregetting ping drops but do not see any issues on the interfaces means things are good and only that the CPU is dropping packets as it is busy processing other packets which have a higher priority.
Also there is a bug related to your issue. The Bug ID is CSCeh45128. The average time may be >1ms.
