This is a bit confusing indeed, since extrinsics in Substrate don't "fail" in the error case.
Extrinsics will emit an
ExtrinsicFailed event when returning an error, but still execute fine.
This stands in contrast to Ethereum, where a transaction reverts and does not execute in the error case.
In general you have to wait for the extrinsic to be included in a block.
Then read the events of your extrinsic from that block. There should either be a
GSRPC has some rough edges, I don't think it exposes everything that is necessary for this approach.
Another "hacky" way of doing it; Wait for the event that would be emitted in the success case with a timeout.
If it times out, you know that it did not work. In the other case it could also be that someone else send a successful extrinsic, whos event you now see.