A Generative Model for Procedural Materials

Paul Guerrero1 Miloš Hašan1 Kalyan Sunkavalli1 Radomír Mĕch1 Tamy Boubekeur1 Niloy J. Mitra1,2
1Adobe Research 2University College London

MatFormer We present MatFormer, a generative model for procedural materials that are represented as node graphs. MatFormer generates an arbitrary number of editable, resolution-independent materials (left), that can be used in realistic scenes (right). All materials in the scene were generated by our method; some of them are shown highlighted in orange on the left. Materials are generated as node graphs (top), where nodes correspond to image operators (each node shows the operator output), and edges control the flow of information between nodes. User can adjust parameters directly using the parameterized material graphs.


Procedural material graphs are a compact, parameteric, and resolution independent representation that are a popular choice for material authoring. However, designing procedural materials requires significant expertise and publicly accessible libraries contain only a few thousand such graphs. We present MatFormer, a generative model that can produce a diverse set of high-quality procedural materials with complex spatial patterns and appearance. While procedural materials can be modeled as directed (operation) graphs, they contain arbitrary numbers of heterogeneous nodes with unstructured, often long-range node connections, and functional constraints on node parameters and connections. MatFormer addresses these challenges with a multi-stage transformer-based model that sequentially generates nodes, node parameters, and edges, while ensuring the semantic validity of the graph. In addition to generation, MatFormer can be used for the auto-completion and exploration of partial material graphs.We qualitatively and quantitatively demonstrate that our method outperforms alternative approaches, in both generated graph and material quality.



Procedural material design (e.g., using a product such as Adobe Substance 3D Designer) typically involves creating a directed node graph, referred to as a material graph. Such graphs consist of a set of nodes — representing noise and pattern generators or operations on textures (e.g., filter kernels, transformations) — and edges — representing the flow of information from the output of the nodes to inputs of the subsequent nodes — finally producing image maps (e.g., roughness, normal, diffuse) for an analytic SVBRDF model. The output maps can be controlled by editing the parameters of the individual nodes. With complex material definitions regularly needing 50+ nodes, authoring such graphs is a form of black magic, limited to a select handful of practitioners. Not surprisingly, the largest publicly-available texture dataset has only a few thousand such definitions, and non-expert users mainly select from these limited options. Hence, there is a demand for automatically generating procedural materials, or assisting with their creation.

In this work, we introduce MatFormer, the first autoregressive generative model for material graphs. MatFormer leverages a transformer-based architecture to model a probability distribution over the space of procedural materials, and subsequently allows sample from this distribution. We found the choice of transformers, as opposed to LSTM, GRU, or graph networks, to be particularly suitable in this context, as transformers effectively handle sparse long-distance connections between graph nodes. However, in order to model the specific structure of material graphs, MatFormer does not generate them in a single pass. Instead, it runs in three sequential stages, each modeled with a dedicated transformer to capture dependencies: first, we generate a sequence of nodes; second, we generate parameters for each of the generated nodes; and finally, we generate directed edges connecting the input and output slots of the generated nodes.


    author = {Guerrero, Paul and Hasan, Milos and Sunkavalli, Kalyan and Mech, Radomir and Boubekeur, Tamy and Mitra, Niloy},
    title = {MatFormer: A Generative Model for Procedural Materials},
    year = {2022},
    volume = {41},
    number = {4},
    doi = {10.1145/3528223.3530173},
    journal = {ACM Trans. Graph.},
    articleno = {46}


We would like to thank Romain Rouffet, Luc Chamerlat, Geoffrey Rosin, and Gaetan Lassagne for their time, suggestions, and helpful feedback.