PETA demonstrators are putting the heat on a North Carolina police department where a cop hanged and choked a K-9 by its leash, and they're demanding justice for the dog. The demonstration went down Friday outside Salisbury PD HQ -- masked up PETA…