I always enjoy returning to the area I grew up in because I have many family members and friends there. I wouldn't mind moving back there some day.
I'd never want to go back to the house I grew up in. It's changed hands a few times and I'm sure it would be too changed for me to enjoy being there.
Same here, I'll hopefully be returning to Michigan and Ohio (Ann Arbor-Detroit-Toledo area) for years to come. I always look forward to those trips, but moving back there? Uh-uh. If I'm going to go through the hassle of moving hundreds of miles (or, who knows, maybe even out of the United States), I'm not going to move to another place with miserable winters. I kind of feel that would be pointless.
The "oldest" house that I remember growing up in, if it were to magically materialize exactly where I wanted to move, it would certainly be too small for me, even after subsequent owners having built on to it somewhat. (The two places I remember before that - a rental farmhouse and, before that, a rental flat in the no-longer-existent Willow Run Village, Michigan...do not exist anymore.)
I did visit it about 6 years ago, and had a good time hanging out with the people there for a while. I even mentioned that there used to be a "pull-down" stairway that went up into the attic, where my room was at the time, and how I wouldn't extend the ladder but go up by using a big cabinet that was next to it. (Our cat Mittens, who had an extra toe on each front foot, used to do the same thing, jumping from the cabinet to the un-extended ladder part when he'd come up to see me. As he jumped he always let out this cute, gentle little "Urrr..." sound.) Amazing...when I showed them where it was, the husband looked and noticed there was very faint evidence of an irregularity in the ceiling that he had
never noticed before, faintly framing exactly where the pull-down stairs had been.
Oh, and thank you Croynan for sharing your thoughts above - very interesting.