I had some backlash from neighbors when I added on my shop, but the only result was I had to state I wasn't going to operate a business in it. Of course I had to comply with the architectural standards in force.
A recent brouhaha came up about whether homeowners could have garden sheds in their yards, with a couple of vehement objections being raised at meetings. The subject was tabled for a "study", so I don't know what will happen on that topic. We also had a case where one homeowner painted his brick, which was technically against the rules although it could be argued either way. That was settled by stating that others could also do so as long as the color was approved by the board in advance.