Matching a Pocket Square without a tie

I have a question about matching a pocket square when I don’t wear a tie, particularly when I wear a sports jacket and nota suit. Should I try to match it with some color that’s in my shirt or go for the opposite, or maybe somethin neutral? I’m looking for a delicate touch and not something that will get all the attention.