And it doesn't mean that some other random group gets torqued off at you; it would (or at least should) mean that individuals involved with your original Vendetta / Debt become interested. Sure, you paid the Yakuza their money, but what about the favor that giving you that loan incurred?? (Note: this is the way that the Yakuza actually work, at least in SR.) Until and unless you repay that favor (i.e. the karma) which may come as a lot of little ones over a period of time, they're going to keep harassing you. Or you wiped out the gang you have a rivalry with; great. What about their friends? Got them too? What about the two or three cops that are now looking for a mass murderer, someone who slaughtered ten or twenty or thirty people, systematically killing everyone involved?
The karma cost can easily be made sense of, if you think about it for a bit.