In his new book SuperSight: What Augmented Reality Means to Our Lives, Work, and the Way We Imagine the Future, author David Rose delves into the current state of augmented reality art, discussing how technology is already transforming countless industries—from food services to medicine to education to Construction and architecture – and what they can achieve in the near future. In the excerpt below, Rose takes a look at two companies that are leveraging computer vision and generative adversarial networks to reimagine current properties as electrified smart homes in the 21st century.
Excerpted with permission from SuperSight: What augmented reality means for our lives, our work, and the way we imagine the future By David Rose, published by BenBella Books.
We should all use solar panels. a period. The average cost of a sustainable energy system has fallen about 70% in the past decade, from $5.86 per watt to $1.50 per watt, so it’s a no-brainer. Without cash, you can fund the installation and start saving a hundred dollars a month the first month, and even more if you live in a sunny south.
Why aren’t we? It is complicated! Mathematics, logistics, taxation, and aesthetics play a role. Many homeowners fear to make their homes as shiny and reflective as Tin Man wizard of oz. The process of figuring out how many panels of the size you need requires learning to “talk about solar energy” in unfamiliar units such as kilowatt-hours. Change always comes with risks, whether actual or only perceived.
The pro-climate mission of Boston-based Energy Sage is to get people to power their homes. That means solar panels on your roof, an electric car, a home battery system, automatic blinds, and a smart thermostat that’s pre-cooled or pre-heated as you drive home. And they’ve partnered with us on Continuum to make potential customers more comfortable with the idea by showing you what an electrified version of their home might look like. Using publicly available Google Home satellite imagery, we digitally resize and overlay solar panels on customers’ rooftops, then show them what their panel will look like from the street and a neighbor’s fence. Next, we take these photos and pair them with data from Project Sunroof, a Google project that helps you get to work on solar potential for your roof. Once you see the beautiful photos of your electrified home and realize how much you’ll save over the years – and have the visual and financial data on hand – it’s a simple decision to go ahead and make that change.
Other home improvement projects will benefit from SuperSight’s envisioned approach. Let’s consider landscape design: another very complex and expensive project with bewildering language, risks, and a dire need for pre-project visualization.
I met landscape designer Julie Moir-Messervy at an MIT competition and immediately became intrigued by her mission: to give homeowners the confidence and tools they need to transform their barren yard into a suite of outdoor living spaces. Her company, HomeOutside, helps people see new possibilities in their backyards using artificial intelligence and computer vision. Once they convincingly visualize their yard, the company makes it easy for them to make that vision a reality by hiring a landscape installer, delivering materials, and even helping spread out payments over time.
Landscaping is not only good for property values; Green plants filter airborne pollutants that cause asthma, help people recover faster from illness, lower summer temperatures, and even reduce crime. A healthy local landscape operates a dynamic system that helps bees and birds, who in turn pollinate the trees and replant the plants. Southwest shade trees can reduce the need for air conditioning, and northeast hedges reduce winter winds and heating bills. More trees means more carbon capture – a ton over the life of each tree – because it absorbs our bad stuff from the air while reducing runoff and erosion.
“Most people don’t do anything in their yards because they don’t know where to start,” Julie told me. “They don’t know what plants to choose and how to arrange them, or they don’t know how to install and take care of landscape design over time.” I was so inspired by working on the problem that I accepted a position on its board of directors and got to work.
HomeOutside trains a Generative Adversarial Network (GAN) to automatically create beautiful, sustainable landscape designs, based on the thousands of designs (think of these as recipes) the company has developed for clients over the past 20 years. The company uses the Google Earth Engine and photogrammetry to start a 3D rendering of any address (US only, for now). The GAN then uses one network (the generator) to make a new design, and another network (the discrimination) to judge or record the work. These two networks continue their iterative game, producing and then scoring, until the spectator decides the landscape has a good composition: shade trees, natural pollinators, grass to play, hardscapes/floors and furniture for gathering places, plant diversity, etc.
Companies that sell solid plants, furniture, lighting, and landscapes are clearly interested in this kind of “fantasy drive” technology, because it bridges the conceptual gap between the current state of someone’s garden and what could be — thus motivating more people to create a dream come true. It’s not only great for homeowners and outdoor retailers – it’s great for the environment, too. But what the company’s environmentally-conscious investors find most appealing about this project is the opportunity to change the landscape of entire neighborhoods on a large scale. What if we could create a new national park across millions of backyards that fusing together places for birds and bees? Each acre of forest absorbs about 2.5 tons of carbon annually. What if we turned neighborhoods into big carbon sequestration zones?
Julie and her team have helped develop HomeOutside’s grand plan to proactively redesign a 70 million front yard, then work with Home Depot, Lowe’s, Wayfair, IKEA and garden centers to email their customers a 3D redesign of their yard. Clients simply step outside their homes, unlock their phones, and through the app’s use of spatial world stabilizers, walk through an immersive animated scene superimposed in their existing backyard. The time-lapse view from sunrise to sunset shows why the edible garden was put into place. Winter visualization illustrates the selection of new fir trees between the yard and the neighbors. Spring flowers bloom with a cacophony of color.
Would people be bothered by the idea of a proactive algorithm redesigning their yards, using new shade trees and naturally pollinating shrubs? It’s not like your front yard is now private, thanks to Google Street View. And if you’re selling your home, you may decide not to hire a landscaper and just opt to post photos of a HomeOutside makeover instead to maximize your attractiveness.
Once this vision technology becomes popular, many different areas will begin to benefit from it. For example, Home Depot recently invested in a startup called Hover, which, after digitizing your home in 3D, visualizes and pricing new paint, siding, and ceiling materials. SuperSight will soon be showing the actual paint crew on their stairs, finishing with the last few brush strokes, so you get that exhilarating experience of the work just finished. Volkswagen may put a new Passat in your driveway, complete with the boats and mountain bikes you know you love at the top. Which company is trying to sell you home and auto insurance? They would expect a disaster scenario: Solar panels fell, a shade tree was struck by lightning, and a new Passat was hit in a hailstorm. It is best to purchase insurance before repainting.
How will we react to these types of immersive designs? With our SuperSight glasses, will we point and position trees, or draw flowers from a set of options, like a 3D version of Photoshop? Will we choose each manufacturer from an extensive list of options for endless control and customization, or will we just tell the system what it likes so it learns our preferences, and then suggest one solution we’ll love? I believe in the happy medium: that we pretty much prefer seeing and choosing from many ‘expertly made’ options, just as we do today when working with an architect, interior designer, or wedding planner.
Experts are usually so good at what they do that it is often a mistake to over-specify certain details. For example, you shouldn’t tell an architect that you want a window exactly here, or an interior designer that you want that particular chair in a certain color in this corner. Instead, express your opinions at a higher level of abstraction (“I want the room to feel more ecologically”) or by describing the desired function (“We want a vegetable garden”), and let them do the detailed work.
The same expert-guided interaction model will dominate our relationships with SuperSight AIs. For the landscape, we might order a more formal French garden with straight layouts and exotic colorful plants, or a graceful organic design that prioritizes privacy from our neighbours. We may refer to a preference for an open space for play, or a scheme filled with more space for a productive garden. As we express these high-level interests, our 3D landscape design will dynamically recalculate to match our preferences. With SuperSight glasses, we’ll be able to test our intuition faster by seeing reconfigurations instantly and in context, superimposed on our real home.
The jury is still out on whether HomeOutside will be able to use this technology to convince millions of homeowners to invest significantly in a sustainable landscape. The test is promising, though; Clients are delighted to see their yards being reimagined and restored. In the next five years, HomeOutside plans to use Google Earth and Street View imagery in an artificial intelligence-generating tool to automatically redesign tens of millions of landscapes using sustainable plants, shade trees, natural pollinators, and bird-friendly berries. If successful, it would mean that 1 million homeowners would plant at least 3 million new shade trees, such as oak and beech, which would each capture 48 pounds of carbon annually as they mature. That’s 14 billion tons of carbon trapped over the lifespan of those trees.
As one HomeOutside consultant summed it up, “You’re building the equivalent of a new national park – our national park! Visualization tools like HomeOutside can convince homeowners to reshape the American landscape.”
This is the ultimate potential power of SuperSight: to help people envision and imagine a future that benefits themselves and the planet.
All products recommended by Engadget are handpicked by our editorial team, independently of the parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.