Dynamic Building System
Dynamic buildings and road system integrated with simulated ecosystem
Overview and Inspiration
Starting with my interactive artwork, Uncertain Associations, I have been working to simulate buildings and infrastructure that interact with a digital ecosystem. Without a strong human element, my dynamic landscapes depict nature devoid of human contact. As most ecosystems, waterways and landscapes have been significantly altered by humans, I want to include abstract cultural representations that interact with simulated environments. In depicting impacts of the anthropocene and climate change, I also want to visualize a range of ways that human communities can interact with ecosystems, including mutualistic relationships and restoration.
I am not interested in depicting individual people or machines, but rather cultural relationships that operate on higher collective levels, such as patterns of urban development and resource extraction systems. I consider that how societies treat the earth is reflected by how we build cities, grow food and move resources through local environments. Where and when we harvest resources, and where we deposit waste into the landscape resembles the life of a larger, emergent organism. More politically, I have often heard indigenous activists speak of how settler society treats the earth mirrors the ways white men treat indigenous women. I also believe that how we relate to the earth reflects how we relate to other cultures, eachother, and ourselves.
These high level, cultural relationships operate on large spatial and temporal scales similar to those of ecosystem adaptation and climate change. I seek to depict cities that represent cultures struggling to adapt to impacts of climate change, visualizing potential futures. In a compressed frame of reference, viewers can see dense urban areas, suburbs, agriculture and resource extraction operations connected by a dynamic transportation network. These components all expand, contract and adapt to changing landscape and climate conditions.
Once again, creating a realistic city is not my goal, but rather to use the idea of roads, buildings and agriculture as metaphors for cultural relationships with nature. Even with simple structural features, I can visualize adaptive strategies to mitigate climate change as well as specific nature based solutions. This is a huge undertaking, and this iterative development will continue through many new projects.
Previous Work
There are many city building video games that simulate complex dynamics within urban areas. Some games, such as SimCity and Cities Skylines, include some relationships to a dynamic landscape, through various disasters such as fires and flooding. However, these interactions are extremely limited, and occur within a reactive landscape that does not change independently of intervention. There are many city building games that include disasters such as flooding and wildfire, but I have not experienced a game where the environment feels alive, with emergent behavior that is influenced by human impacts. Lichenia, an experimental dystopian city building game, employs a cellular automata approach to create a dynamic landscape where urban, agriculture and ecological elements change unexpectedly. This system is interesting to experiment with, and is certainly an example of Video Game Art, but is largely inaccessible and unintelligible. There are also many procedural city generators, but these do not support dynamic growth and decay of buildings.
Technical Approach
After much planning and deliberation, I have decided to simulate cities through different agent based systems that interact with GPU based data structures. Human relationships with the virtual environment is expressed through, point, linear and block actions, similar to those used by Geographic Information Systems (GIS). Footprint geometry for buildings and roads are rendered to 2D data textures from a top down camera that covers the entire landscape. This separate render texture can be applied to vegetation rendering and update shaders to remove grass, bushes and trees. Rendering high precision data with a specified elevation value, anthropogenic features can also be used to change elevation data. Clearing vegetation and changing landscape forms are basic interactions that can be used to depict cumulative impacts of resource extraction and development.
To represent structures that decay over time, road and building footprints are rendered to a separate render texture that gradually decrements. As corresponding values for infrastructure get lower, the rendering of roads and buildings can be degraded until they dissolve entirely. As roads and building foundations decay, they impact vegetation and soil less severely, allowing for regrowth. Buildings and road data can also be degraded with water, increasing with flow rate and depth. Similarly, data for buildings can be impacted by wildfire, allowing for them to be burned if they are counted as fuel. Future impacts to buildings can include landslides and extremely high wind (hurricanes, tornadoes). Infrastructure that expands, then decays is crucial to visualize various impacts of climate change.
To create a dynamic system that allows for urban renewal and adaptation is much more difficult. To start, I am treating individual buildings as the base artificial-life unit, where each structure senses its own condition, and decides to restore itself if it has the energy to do so. Each building will simulate a basic energy economy that interacts with the economy of the urban area. If a building has enough energy to repair damage, its data values are refreshed, restoring decay values in the structure data texture. If a building is too damaged for repair, it continues to decay until the building has disintegrated. Buildings can also be demolished or salvaged if they are beyond repair. This basic system involves a third render texture that road and building footprints are imprinted on, representing the index of each element. Read back into the CPU, the total number of pixels represented in landscape data can be accumulated in statistics for individual buildings. With this total pixel value, total and average health values can be determined for individual buildings.
Buildings and roads under construction and corresponding render textures. Top Right texture applies structure data for construction and repair to lower right texture, which gradually decreases overtime as decay.
Refinement
At first, my buildings would repair themselves even if they were actively on fire, which is not desirable. To avoid this phenomena, I implemented a damage value that is increased with fire and flooding, and gradually resets over time. This value is read back into the CPU along with foundation and road values, and is calculated as a part of building statistics. When a building is beyond repair and completely disintegrated (0 health) it can be removed from the CPU list of buildings. This free cell can then be redeveloped at the block level, depending on the balance of its current economy. A decayed building can be rebuilt with different materials, a different size or become an agriculture cell or stormwater basin. As the number of buildings and resource cells change, each block can adapt with shifting priorities.
Buildings are represented by footprint quads, where roads can be represented by linear triangle strips. Each form can be animated with a buildTime value, and repaired with a refreshTime value. For the data transfer shaders, pixels are clipped if they are not within a couple seconds of the build or refresh times. This refresh step can also be applied to continually remove vegetation and repair eroded landforms. This system was carefully developed to also accommodate other infrastructure such as roads, stormwater networks, irrigation systems as well as agricultural units in future developments.
A more naive approach would be to represent cities and agriculture through a cellular automata system, similar to Lichenia. I plan to experiment with this strategy for more simple projects, but I do not believe it would be feasible to represent individual buildings, roads and gardens that are important for a medium scale landscape. Additionally, architectural variation and development patterns would be much more limited, restricting potential depictions of cultural diversity. However, to render a very large landscape, a more simple, purely cellular automata approach may be more effective.
Interaction Diagram for dynamic building system
Artistic Considerations
With urban impacts to the landscape, as well as ecological impacts to cities, it is possible to create many dystopian landscapes. While developing these systems in the summer of 2023, I was anxiously seeing images of massive floods, devastation from war in Ukraine and wildfire impacted cities in Hawaii and Canada. I started to see shadows of my simulations in the real world, especially with Lahaina as direct source material for this work from an earlier visit in 2022. Even before getting to the point of rendering such landscapes, I knew that depicting major disasters through art is extremely sensitive. However, I feel that it is important to communicate about the impacts we face through an abstract timescale on higher levels, but I do not wish to retraumatize or desensitize anyone to the grim realities of our current time.
Before ever showing this work, I knew that I needed to depict regrowth and restoration of cities and agricultural areas impacted by disaster. I want to depict individual buildings as artificial-life entities, and cities as larger units which adapt through actions of distributed cells. In reality, cities have always been rebuilt, and it is in this stage that there is the most potential for change with different patterns of development. Again, operating on a high level above individual perception allows for conceptual distance between lived experience. There are many artists living in frontline communities that are doing excellent work to tell stories about devastation, adaptation and resilience year to year all over the world. Offering a long-term perspective and a systems understanding of climate adaptation is an additional way of thinking that can support more nuanced thinking about the future.
This kind of simulation work is very complicated, and potentially limitless in its detail and fidelity. I have worked carefully to create an efficient and scalable system to create dynamic buildings. Potentially, individual users could interact with a small area of space on their personal device that impacts on cells in a larger, collective landscape. I have long since thought about an art installation where the gallery computer would act as a central server, simulating ecological dynamics, and receiving input data from remote viewers who impact the landscape in small ways. Viewers in the gallery space would experience and interact with the virtual landscape in a different way and would witness collective interactions of viewers interacting through personal phones and computers. It is feasible for viewers to create a building with aesthetics of their choice, and determine what pixels correlate to gardens, concrete or water management features. In such a system, neighbors would significantly impact each other, especially with flooding, pollution and wildfire.
Many other interfaces may be used to interact with a dynamic urban environment. As a first foray with an artwork featuring a simulated urban community, Hydrosheds allows viewers to change building density and surface conditions, which dynamically affect water flow and vegetation. However, Hydrosheds is essentially a static urban simulation with a reactive ecosystem, as buildings do not decay and are not impacted by flooding, and vegetation does not grow or adapt. Future work will be conducted to create an interactive artwork where viewers can influence an urban community that dynamically changes with a virtual ecosystem impacted by climate change.
