Generative AI Models for Different Steps in Architectural Design: A Literature Review

3 Generative AI for Architectural Design Process

In this paper, it was found that the application of generative AI in architectural design focuses primarily on specific architectural tasks, categorized into concept image generation, architectural 3D form generation, plan generation, facade generation, and structural system generation, as previously mentioned. This paper analyzes and summarizes the application methods for each architectural task, examining input data, output data, application scenarios, and technological approaches.

3.1 Architectural Concept Design

Table 1: Application of generative AI in architectural concept image generation.

Concept is defined as “the figure of an object, along with other representations, such as attributes or functions of the object, which existed, is existing, or might exist in the human mind, as well as in the real world” [150].Concept also refers to the mental representation that the brain uses to denote a class of symbols that are inferred from the physical material [10]. Many architectural concept expressions synthesize design elements into 2D images, thereby reflecting the architect’s personal style and experience.

Application of Generative AI in Architectural Concept Image Generation:

The applications of generative AI in architectural concept generation include four main categories, as shown in the Table 1: 1) generating architectural concept images based on text or images; 2) transfer of architectural concept image style based on images; 3) generating concept images based on semantic images; 4) generating semantic images based on concept images.

First, [75] use linear interpolation techniques to generate architectural images from text across various perspectives, while certain authors’ [48, 79, 112, 181, 148, 143] use of direct generation from textual prompts simplifies the concept image creation process. [12] exploit the stable DM to generate architectural interior images. Several authors [14, 156, 71, 18, 133, 13, 60, 189, 58] use generative AI tools to create architectural design concept images based on image and text prompts. [152] incorporated Midjourney into design courses, guiding students to use orthographic projections as input to generate conceptual images.

Second, [170] used a GAN model to colorize line sketches of urban block layout images. [44] exploit GAN models to generate comfortable underground space renderings from virtual 3D space images; [135] utilize GANs to facilitate the creation of interior decoration images from 360-degree panoramic interior images. [120] and [168] utilize GANs to generate architectural images based on sketch line drawings. [80] explores the use of GANs and DDPM for style transfer on food images, converting them into creative architectural images.

Third,[72] use GANs to generate urban street images based on image semantic labels (masks), allowing precise control over the content of the generated images. [46] use VAE to produce semantic images corresponding to architectural images.

3.2 Architectural 3D Forms Design

According to [19] , “in art and design, we often use the ‘form’ to denote the formal structure of a work the manner of arranging and coordinating the elements and parts of a composition so as to produce a coherent image.” Architectural 3D form design refers to the initial stage of designing 3D conceptual models or representations of buildings or structures. This phase focuses on exploring and developing basic spatial configurations, volumes, and massing of the architectural design before detailed refinement.

Table 2: Applications of generative AI in architectural 3D forms generation.

Applications of Generative AI in Architectural 3D Forms Generation:

The applications of generative AI in this process include four main categories, as shown in the Table 2: 1) generating architectural 3D forms based on parameters or text, and classification of preliminary 3D forms; 2) generating architectural 3D forms based on images (usually from sketches, color patterns, floor plans, etc.) and voxel volume; 3) generating images with 3D form information; 4) generating environmental performance evaluation based on 3D forms.

First, generative AI facilitates the generation of preliminary 3D forms based on input parameters or the conduct of classification analysis. Initially, [23] use VAEs for generating preliminary 3D forms from input parameters. Building on this, [187] apply GANs to refine the process by training point cloud data of 3D models and using category prompts for more precise reconstructions. [191] employ the approach that facilitates the creation of innovative architectural 3D forms using input interpolation. [91] utilize diffusion probability models to provide a unique training method for Taihu stone and architectural 3D forms and discover transitional forms through interpolation. [117] use a structural GAN model that uses point cloud data to generate 3D models based on parameters such as length, width, and height. [131] exploit VAEs to generate 3D voxel models guided by textual labels. [97] apply interpolation to create new 3D elements with transitioning forms. [32] operate 2D images with 3D voxel information, which can be produced using the input RGB channel values. [73] use 3D-GAN to generate voxelized and point cloud representations of building 3D forms components in accordance with textual labels. [74] employ the 3D adversarial autoencoder model to train point cloud representations of a 3D model, thereby facilitating the classification of architectural forms. [49] utilize VAE to train signed distance function (SDF) voxels and to conduct clustering analysis on latent vector representations of 3D models. [192] present a novel method to generate urban block depth images from textual descriptions using latent DMs, particularly stable diffusion, and reconstruct 3D urban models from these depth images.

Second, generative AI involves using images as the generation conditions for generative AI to produce 3D forms. [153] utilize the integration of VAE and GAN models to facilitate the generation of architectural 3D forms from sketches. [109] operate CGAN to create architectural 3D models from design concept sketches, and [77] generate 3D models from a singular concept sketch combined with environmental images. [132] utilize diffusion probability models to train 3D models, introducing noise into the 3D voxel volume to create novel forms. [6] employ StyleGAN to generate detailed 3D forms from architectural floor plans. In addition, [174] use CGAN to generate detailed 3D forms based on spatial sequence diagrams and spatial requirements.

Third, training on 3D model data is more challenging than that on 2D image data. Researchers have simplified the training process to address these challenges by converting 3D forms into 2D image representations, such as grayscale images enriched with height information. [176] exploit the practice of transforming 3D models into section images for reconstruction, followed by reverting these section images back to 3D forms, thereby significantly reducing both training duration and costs and ensuring accurate restoration of the original 3D models. A few authors [121, 188, 82, 157, 64] use this method for the generation of architectural 3D forms tailored to specific sites and their surrounding environments.

Fourth, [5] and [59] exploit generative AI to conduct site and architectural environmental performance evaluations based on 3D models. This involves generating images for assessments, such as view analysis, sunlight exposure, and solar radiation, among others.

3.3 Architectural Plan Design

An architectural plan is a horizontal section taken at approximately different levels [159]. Architectural plan design includes building floor plan design and layout design. Building floor plan design refers to the layout or plan view of a specific structure on a horizontal plane. It depicts the detailed arrangement of different levels of the building (typically ground floor, upper floors, etc.), including the positions, dimensions, and relationships of rooms, corridors, walls, doors, and windows. Further, architectural floor plan design emphasizes spatial functional zoning, scale proportions, room uses, and other aspects , making it a fundamental and crucial component of architectural design. Layout design typically refers to the broader arrangement and configuration of spaces or sites, not limited to the interior of buildings. It can encompass the positioning of buildings on a site, relationships among buildings, and the arrangement of other elements such as roads and landscapes within the overall environment. Layout design focuses on optimizing space utilization, enhancing functional efficiency, and meeting design requirements and environmental conditions while considering the interaction between buildings and their surroundings.

Table 3: Application of generative AI in the architectural plan generation.

Application of Generative AI in Architectural Plan Generation:

The applications of generative AI in architectural plan generation include four main categories, as shown in the Table 3: 1) generating building floor plans based on 2D images, usually from building footprints, heatmap of environmental performance evaluation, functional space layout, spatial sequences, and floor plan; 2) generating functional space layout based on 2D images , usually from functional space layout, spatial sequences, building footprints, requirements and standards, site conditions, and surroundings conditions; 3) generating spatial sequences based on 2D images, usually from building floor plan, building footprints, and environmental performance evaluation heatmap; 4) generating spatial environmental performance evaluations heatmap based on 2D images, usually from building floor plans and functional space layout.

First, in generating architectural floor plans, certain authors [38, 103, 15, 160, 11] create a functional space layout diagram from building footprints or site boundaries and then develop an architectural floor plan, progressing from building footprint or site boundaries to functional space layout, and finally to floor plan. [2] use generative AI models to convert microorganism drawings into architectural floor plans in Palladio’s style. [68] and [149] utilize GANs model to refine architectural floor plans, obtaining plans with flat furniture layouts. The generation and reconstruction of floor plans are achieved by [154] through training on architectural floor plan datasets. [61] generate floor plans based on the heatmap of spatial environmental evaluation, such as wind and lighting conditions. [4] employed CycleGAN to perform bidirectional conversions between Istanbul’s historical Pervititch maps and plan views derived from modern satellite imagery, facilitating plan generation based on the transformed views.

Second, generative AI plays a various roles in the generation of functional space layouts. [167] exploit generative AI to reconstruct and produce matching functional layout diagrams based on the implicit information within the functional space layout. [169] generate street layout based on functional space layout. Certain researchers[57, 39, 90] generate or complete incomplete functional space layouts based on specific demands. Many researchers [107, 108, 164, 155, 96, 134, 1] use spatial sequence diagrams to generate functional space layout. Moreover, several authors [146, 93, 92] use generative AI to generate functional space layouts based on the designated site boundary. [183] and [113] skillfully use building footprints as conditions to generate a functional space layout. [47] proposes a graph-based deep learning model called graph2pix (G2P), which incorporates room area and type information into the graph’s nodes (bubble chart) to generate floor plans that meet specific user requirements. [173], based on DDPM, introduce a multi-conditional generative model called FloorplanDiffusion; by inputting bubble charts, building footprints, and designer requirements, the model generates diverse, high-quality, and controllable residential floor plans. [84] introduces ChatDesign, a method utilizing pre-trained LLMs to generate functional space layouts from natural language descriptions. [98] use DCGAN and StyleGAN to convert satellite image datasets into urban functional layout images.

Third, generative AI models demonstrate exceptional performance in the generation of spatial sequences. [29] utilize EdgeGAN to identify and reconstruct wall layout sequences from floor plans. [66] employ isovists to predict wall sequence diagrams. [142] use a DDPM to generate spatial sequence diagrams based on specific boundaries. Fourth, [30] utilize CGAN to foresee space environmental performance evaluations from floor plans, such as light exposure and isovists. [105] utilize pix2pix to predict indoor brightness and [50] utilize pix2pix to predict solar radiation based on functional space layout diagrams.

3.4 Architectural Facade Design

Architectural facade design refers to the process of designing the exterior appearance of a building, specifically focusing on the facade or the outer shell. The facade plays a crucial role in the overall aesthetic appeal of a building and often serves functional purposes such as providing weather protection, insulation, and visual identity. Facade design involves considerations of materials, textures, colors, proportions, and architectural styles to achieve the desired visual impact while meeting functional requirements [54].

Table 4: Application of generative AI in architectural facade generation.

Application of Generative AI in Architectural Facade Generation:

The applications of generative AI in architectural facade design include three main categories, as shown in the Table 4: 1) generating facade images based on semantic segmentation maps; 2) using generative AI models for facade image style transfer; 3) generating semantic segmentation maps of facade based on images.

In generating facade images, many researchers [172, 21, 144, 180, 89, 178, 31, 65] utilize generative AI to create architectural facade images based on semantic segmentation maps that precisely annotate the form and location of elements such as walls, windows, and other components. In facade style transfer, [28] use CycleGAN to train facade datasets for generating novel architectural facade images. [147] and [22] apply style transfer to architectural facades by incorporating style images. [99] facilitates style transfer between facade images to generate a new facade image. In generating semantic segmentation maps of architectural facades, [9] utilize GANs to reconstruct these maps, including rebuilding occluded parts from unobstructed areas. [160] utilize pix2pix to complete facade mask images for all four building directions, which can be generated from images of the outline of the architectural facade.

3.5 Architectural Structural System Design

Architectural structural system design refers to the process of designing the structural framework and system that sup- ports a building’s architecture. It involves determining the type of structural elements (such as beams, columns, slabs, and walls), their arrangement, and their integration with the architectural design to ensure structural stability, safety, and functionality of the building [3].

Table 5: Application of generative AI in architectural structural system generation.

Application of Generative AI in Architectural Structural System Generation:

The applications of generative AI in structural system design primarily involve the prediction and optimization of structural layout and dimensions. The application methods are illustrated in Table 5. In predicting structure layout, several researchers [86, 36, 37, 81] utilize generative AI to recognize architectural floor plans and generate structural layout images. A few authors [94, 85, 35, 185] exploit generative AI to generate structural layout diagrams based on specified structural load capacities. Generative AI can refine existing structural layouts. [87] use GANs to propose seismic isolation solutions for walls, and [186] use pix2pixHD to combine functional space layout and floor plan to create the corresponding structure layout. [43] propose an intelligent shear wall layout design method based on DMs. In optimizing structural layout, [34] apply StructGAN-KNWL to forecast and create more suitable structural sizes and layouts based on existing structural layouts.

3.6 Architectural Section Design

The building section is the premier drawing for revealing and studying the relationship between the floors, walls, and roof structure of a building and the dimensions and vertical scale of the spaces defined by these elements [20].

Application of Generative AI in Architectural Section Generation:

The applications of generative AI in architectural section design include three main categories, as shown in the Table 6: 1) spatial feature extraction and construction from section; 2) style transfer and form generation from section; 3) architectural restoration and reconstruction from section. First, [26] use Pix2Pix to generate sequentially stacked section drawings of Taihu stone, extract spatial variation patterns, and construct a 3D form reflecting Taihu stone characteristics. Second, [175] decompose 3D architectural models into 2D sequentially stacked section images and applied pix2pix and cycleGAN, to modify the style of the section drawings.They generated new stylized section images and used them to construct 3D architectural forms, ensuring pixel continuity to preserve the model’s structural and spatial integrity. Third, [45] use pix2pix to train on collected kümbet section drawings and employed SSIM (Structural Similarity Index) to evaluate image similarity, facilitating the prediction and reconstruction of missing architectural components.

Table 6: Application of generative AI in architectural section generation.