Sketch2Terrain: AI‑Driven Real‑Time Terrain Sketch Mapping in Augmented Reality

Sketch mapping is an effective technique to externalize and communicate spatial information. However, it has been limited to 2D mediums, making it difficult to represent 3D information, particularly for terrains with elevation changes. We present Sketch2Terrain, an intuitive generative-3D-sketch-mapping system combining freehand sketching with generative Artificial Intelligence that radically changes sketch map creation and representation using Augmented Reality. Sketch2Terrain empowers non-experts to create unambiguous sketch maps of natural environments and provides a homogeneous interface for researchers to collect data and conduct experiments. A between-subject study (N=36) revealed that generative-3D-sketch-mapping improved efficiency by 38.4%, terrain-topology accuracy by 12.5%, and landmark accuracy by up to 12.1%, with only a 4.7% trade-off in terrain-elevation accuracy compared to freehand 3D-sketch-mapping. Additionally, generative-3D-sketch-mapping reduced perceived strain by 60.5% and stress by 39.5% over 2D-sketch-mapping. These findings underscore potential applications of generative-3D-sketch-mapping for in-depth understanding and accurate representation of vertically complex environments. The implementation is publicly available.

Recommended citation: Tianyi Xiao, Yizi Chen*, Sailin Zhong, Peter Kiefer, Jakub Krukar, Kevin Gonyop Kim, Lorenz Hurni, Angela Schwering, Martin Raubal. (2025). Sketch2Terrain: AI-Driven Real-Time Terrain Sketch Mapping in Augmented Reality. In Proc. of the ACM Conference on Human Factors in Computing Systems (CHI 2025).
Download Paper