In some ways, the state of suburbia can be thought of as a response to technological developments. Which is to say, suburbia would not exist in its current form if car technology had not made low-end cars so cheap to produce and the modern suburban home so easy to produce. The principles of New Suburbanist planning are good, but a real, radical change in the future of home ownership will lie in an extreme advancement of prefabricating technologies (one example is 3D printing), since that would create a necessary market shift if the cost is lower.
Here's a thought experiment: if a revolutionary technological development in yacht development caused 100-foot yachts to be produced for as low as $10,000 a yacht, what would happen? One, everyone near an ocean would own one. But more importantly, you would see a rise of communities that exist solely by transportation-via-yacht.
Here's an interesting truth about regulation: if you regulate strongly enough, such as in the case of illegal drugs, it creates profit incentives for people to go outside of that regulation. (And also creates an extra cost of enforcement, but that's another story.) As much as 42 percent of cigarettes in New York are black market cigarettes. You can nudge people into making different choices, but if people want something badly enough (sex, alcohol, drugs, whatever) you can't force them to not want it.
This means: the cheapness of cars cannot be taxed away, since eventually you would create a black market for vehicles. Alternatively, you would create an incentive to move to another state where vehicles are not so highly taxed. But if this regulation occurred on a federal level, you would eventually create a black market. Sure, if the difference between a black market car and a normal one is $2,000 versus $5,000, many people will not want to take the risk. But if the difference in cost between a black market car and a legal one was extreme -- say, $2,000 versus $20,000 -- you'd get the same attitude that people have toward pirating Adobe Photoshop. Sure, you could end the mortgage interest deduction, which I agree with, since it creates a bubble similar to what student loans do to college tuition; sure, you could build less "highways to nowhere" and spend that instead on high-speed rail; at the end of the day though, it won't dramatically effect suburban development, except maybe in a decrease of McMansion development.
In a neighborhood near my old university, a developer built a large high-rise for luxury condos. Many of these have been vacant for quite some time because there isn't a good incentive to buy one: for the price of a condo there, you could get a much roomier house in a similar location. Sure, people who like dense living will buy them, but a lot of people won't. And until the dense areas become such hubs of activity that the incentive of living there is worth the tradeoff, the non-New Urbanist home buyer has the luxury to consider alternative options.
I believe you can draw an analogy between marketing condos to people in non-dense environments and marketing laptops to teenagers in 2003-2004. That is, laptops only became viable for most teenagers to own after the technology developed to a point where they could do everything a desktop could do. Before that, you only bought a laptop if you had some reason to trade computing power for portability. Similarly, in a lot of cities, you will only live in a dense environment if you have some sort of trade-off going on: you are planning on being there for less than five years, the decrease in living space is worth the location, and so on.
Here's another thought experiment: what would a greater incentive to live in condos or denser housing arrangements look like?
Probably something like these prefabricated condos, except crappier looking.
Cheap, prefabricated housing already exists in the form of trailer parks. People don't like talking about that though because the kind of people likely to advocate for "prefab" housing are also the kind of people likely to lump in trailer park-dwellers as the wrong kind of white people. But there are tradeoffs: lack of security, for one. A trailer is a far greater liability, as far as your property safety is concerned, than a house is. Additionally, you probably won't have a dedicated shower/bathroom, which is a dealbreaker for a lot of people.
But if the technology advances far enough, you could have cheap, prefabricated housing that takes care of the bathroom/shower/security issue.
Professor Behrokh Khoshnevis estimates that, using 3D printing, a 25,000 square-foot house could be built in 24 hours, optimistically speaking. How much this would cost is another matter,but if the design is standardized enough, presumably not a lot.
To date, two-story buildings have always been more expensive than one-story buildings. This a painfully obvious observation. But if the technology in prefabrication advanced to the point where the difference between a two-story and a one-story was negligible -- that is, the difference between a laptop and a desktop for most PC users nowadays -- condos may be a far more affordable option than dedicated houses. Alternatively, this would make the single family tower thing less of a mockable pipe dream.
There is a tremendous amount that can be accomplished by zoning, by new development techniques, by New Suburbanism / New Urbanism-lite. Plum Creek is a really good example. (Admittedly though, I've heard it suffers from HOA nazism re: lawn maintenance.) But zoning can only nudge people so far, and human behavior can't be regulated cold-turkey. Suburbia is in part a development of the highway system, yes, but technological development cannot be ruled out, since it's also a factor of the cheapness of cars in general. The kind of radical, nation-wide change that will edge people out of suburbia in a massive way is only likely to happen via the same forces that caused the shift from desktops to laptops, or the forces that caused suburbia to exist in the first place: by revolutionary technological development that minimizes unwanted tradeoffs and radically shifts infrastructure and consumer buying choices.