The metadata provides essential information regarding the substrate blockchain you are connecting to.
It is unique to that chain's runtime.
The chain's runtime metadata includes information such as storage items, extrinsic calls, events, constants, and errors that are exposed by that chain's runtime via the various pallets that were used to construct it.
The metadata most likely will change if the runtime gets upgraded. Therefore you need to make sure you always get the latest metadata.
The frame-metadata repo defines the metadata. Substrate chains implement that type definition. Frame-metadata type definitions are versioned e.g. v12, v13, ...
You can see a nicely formatted example of a metadata here.
I do see your logic though. There could be a way of caching the metadata on the client side so that subsequent requests only check the metadata's version number/hash with the one you have cached. If the version is the same, no need to get the full metadata. If the version you have is out-of-date, then pull the full metadata from the chain.