Yes, class-map is supported in multi-context mode.
You should see it in each context (not in system context) in config mode:
ciscoasa/admin(config)# cla?
configure mode commands/options:
class-map
ciscoasa/admin(config)# class-map ?
configure mode commands/options:
WORD < 41 char class-map name
type Specifies the type of class-map