Hugging Face introduced a method of training models to draw through code

The team at Hugging Face has published a methodology for training language models to generate watercolor images using code. The process is based on the use of the TRL (Transformer Reinforcement Learning) library and the OpenEnv environment, which allows agents to interact with graphics engines.

The approach is based on converting drawing commands into token sequences that the model generates to control a virtual brush. The system uses PPO (Proximal Policy Optimization) to fine-tune the model based on rewards, where the target function is the visual similarity between the generated image and the given reference. In the experiments, the model was trained on a dataset containing 5000 unique strokes and color palettes.

The technical implementation includes the integration of OpenEnv to simulate a canvas, which allows the model to receive real-time feedback. The use of TRL ensures the stability of the training process when working with discrete actions, such as selecting coordinates, brush pressure, and pigment transparency. The training was conducted on a cluster using 8 GPU NVIDIA A100, which allowed the algorithm to converge in 12 hours.

This method demonstrates the possibility of controlling visual content through text models without using diffusion processes. The developers note that this approach allows for the creation of interpretable images, where each element is the result of executing a specific program instruction. The toolkit is available for integration into existing reinforcement learning agent training pipelines.