AI for Game Development: Creating a Farming Game in 5 Days (Part 1)

TL;DR

Hugging Face has released a guide on integrating AI tools into the game development workflow, beginning with using Stable Diffusion to generate concept art and establish a visual style for a farming game. This approach allows developers to rapidly prototype visual directions and translate AI-generated imagery into a functional Unity scene.

Establishing Art Style with Stable Diffusion

Stable Diffusion, an open-source text-to-image model, is used to define the game's visual identity. By generating concept art, developers can create a reference point for the entire project's aesthetic before beginning technical implementation.

Deployment Options for Stable Diffusion

Depending on hardware availability, Stable Diffusion can be deployed in two ways:

  • Local Installation: Using the Automatic1111 WebUI. This requires an Nvidia GPU with at least 8GB of VRAM, Python 3.10.6, and git. The process involves cloning the repository, installing weights (specifically Stable Diffusion 1.5), and running the webui-user.bat file.
  • Online Solutions: For those without compatible hardware, options include Hugging Face Spaces (such as the Stable Diffusion 2.1 Demo or camemduru webui) or using the Diffusers library to build a custom solution.

Prompting Strategies for Game Concept Art

To achieve reproducible and high-quality results for game assets, the guide recommends specific prompting techniques:

  • Form Constraint: Use keywords like "isometric," "simple," and "solid shapes" to create styles that are easier to replicate within a game engine.
  • Artist References: Incorporating the names of specific artists is a highly effective way to guide the model toward higher-quality, specific styles.
  • Keyword Optimization: Avoid terms like "low poly," which may degrade the quality of the output despite being conceptually relevant.

For this project, the final prompt used was: "isometric render of a farm by a river, simple, solid shapes, james gilleard, atey ghailan."

Implementing AI Concepts in Unity

Once the concept art is generated, it serves as a visual blueprint for the scene construction in the Unity game engine. The implementation follows a structured pipeline:

  1. Project Setup: The project uses Unity 2021.9.3f1 with the Universal Render Pipeline (URP).
  2. Scene Blocking: The environment is initially built using basic 3D shapes (e.g., cubes) to establish layout.
  3. Material and Lighting: Materials are applied based on the concept art. Lighting is configured with a warm sun (#FFE08C, intensity 1.25) and soft ambient lighting (#B3AF91).
  4. Camera Configuration: An orthographic projection is used for the camera to match the isometric perspective of the AI-generated concept art.
  5. Final Polish: The scene is completed using a Stylized Water Shader from the Unity Asset Store and post-processing settings (ACES tonemapping and +0.2 exposure).

Sources