I created a house with the 3D layer option enabled and rotated the sides and roof. When I went Move with Scaling to Maintian Size, only the the front of the house maintainted its shape. The side walls and roof elongate.
It is not clear to me what you did to set up the scene.
Have you used the 4 objects to construct a 3D house in 3D space?
Can you show the layer in the Timeline?
I believe the Maintain Size feature is intended for single or multi-plane background scenes not 2D images assembled into 3D objects. Once you have the plane at an angle the Maintain Size effect is limited because of the vanishing point convergence factor. Maintain Size exists so you can adjust Z depth which opens up an opportunity for parallax in your camera movement without having to deal with accommodating the effect it would otherwise have on the relative size of each 2D plane.
It may not be stated but the illustrations in the documentation imply that the images Maintain Size are applied to should be perpendicular to the camera.
Maintain Size is an amazing feature. Being able to move images on the Z axis without it effecting the perception of their relative size is incredibly useful.
What you are trying to do may have to be faked. You may need to make 2D images of the house for use in certain situations. Reserve this house made into a 3D box for instances where you need to navigate the 3D space. Use the perception of 3D for when you need to maintain the relative size and can get away with the illusion of 3D.