cumulus_pallet_xcmp_queue just an implementation for
HRMP (relayed messages b/w sibling parachain) or is it the actual implementation of
XCMP (Direct p2p messages b/w sibling parachains) ?
The XCMP queue pallet in cumulus gets its message passed in from the relay chain via the para-inherents validation data. It is not doing direct para-to-para messaging but always uses the Relay chain in between (often referred to as "XCMP-lite").
The outbound messages from the queue are copied to the collation info by the parachain-system pallet at the end of every block. The Relay can then extract the messages when enacting a candidate here and route them accordingly.