Am I wrong?

Today a neighbor who lives around the corner came over and asked if he can put a campaign sign in our yard for a candidate in a local election. This candidate is apparently a personal friend of his for over 10 years. However, we are supporting his opponent who we know and feel has done a good job. I felt as a good compromise, I told him that he can place the sign but at the far corner of our lot. As he was placing the sign in the direct center of the lot, I advised him that we were not in support of his candidate, but would be willing to allow it at the corner for the visibility. He copped an attitude and spouted something of well then we didnt need the sign we wouldnt want to make our favored candidate mad and that he must be some close personal friend or something. I was trying to compromise by allowing his candidate some exposure but not directly showing that it is who we personally supported. We live on a buzy street and it would have been seen either way. Am I wrong here?