There are many good reasons for picking on a non-leader.
- The non-leader may have specific resources which you hope to steal.
- The non-leader may be competing with you for longer road or most soldiers, while the leader isn't.
- The non-leader may be able to block you unless you starve them of resources
- The non-leader may have previously been the leader, and the current player simply didn't notice the change.
- The non-leader may be deliberately positioning themselves behind the leader - building roads, but not yet towns, accumulating cards, which may be scoring cards, and the classic case of building two separate roads separated by one space, waiting to become the longest road at the right time.
I tend to find the last of these cases tend to be the most vocal complainers - "Why are you robbing me, I am not the leader". Where I can see it is deliberate, I rob them even more, and mock them.
But at other times, if I rob a non-leader there is a good reason for it, and I make no apologies. I believe in "fair play", but playing lip service to this "unspoken rule" when it is detrimental to my point-scoring is just silly.