If a judge considers it "transformational", then no (ie if it is simply a part of the larger image of your house). If it occupies teh majority of the scene (ie you add your door frame as framing), then yes, it is infringment.
This has become an issue with statues in public places. For some reason I don't understand, the sculptor owns the copyright for the statue, even in public places. Several scupltors ahve sued and won cases where photographers made money selling photos of the statues, and even photos where the statue is not the promenent part of the image (ie an image of a park or town square where the statue is central to the park or square).
It is complex and not always consistent; every country has different rules - in Britian a photographer won an infringement case because a photo of his was 'recreated', a b&w backdrop of buckingham palace with a colour red double decker. Because the new image looked too close to the 'original', the second photographer was found to be infringing. This has set a bad precedent in Britian where one man now essentially "owns" the idea of a red bus in front of a b&w buckingham palace.