[Feb-Mar] I turned a drawing program into a video game engine

[May 2026: Article is being actively completed. I am compiling footage and editing my script(s) to add here as demos.] Fantasy Life i: The Girl Who Steals Time is a delightful game. I am a huge fan of their small franchise and fought tooth and nail to activate the .NET 3.5 Framework on Windows to be able to play the game on my computer. In under a month, my more relatively-previously-unused XBOX controller developed stick drift. I had grown familiar and accustomed to using this piece of technology, and when I opened my personal favorite DCC, Krita, while my XBOX controller was connected to my laptop, I thought: Can I use command inputs from my XBOX controller to assist in working in a drawing program? From a layman's perspective, drawing programs are able to use moderately unconventional equipment for controls. For example, a drawing tablet - especially one with fancy buttons - is a particularly unique interface interaction tool, and I have convenient hot keys assigned to my associated drawing tablet pen.

Unfortunately, a video game controller is neither a keyboard nor a drawing tablet, and therefore had no internal or keybinding support.

Now, you see, the reason I love Krita so much is because in its' open-source nature it provides a lot of room for technical users to create, integrate, or modify elements of the program. And by this point I had begun exploring what I could create using Krita's internal Python lib and light Qt implementation to create custom scripts and dockers. Though Krita Python allowed me to run canvas-editing commands, Krita's Qt implementation was light enough wherein I could not create something directly in Krita that could track my video game controller for its inputs. Being inspired by some work demonstrated to me by Christopher Jarvis, I concluded that I could use ✨sockets✨. In other words, I could create an external program that tracked my video game controller inputs and pipe them over to Krita, which would in turn run commands depending on what I did with my controller. My result so far has been two scripts: I have one running in Krita that receives my controller inputs and runs associated commands, and one that I'm running in PyCharm that visually shows me what keys I'm hitting on my controller while sending the results to Krita.

This is the part that has turned Krita into a "video game engine":

The controller joystick sends command(s) to Krita, and for each "frame", the "game character" layer in Krita is either moved, or clears-and-redraws in the direction indicated by the controller.

A game-specific controller can be used to move a character across a canvas in the digital art software. X/Z positioning, brush radii, color & selection detection tools, the presence of several layers, and canvas modification capabilities provided by Krita's Python allow for procedural interaction that could let one create a fully-functioning video game in Krita. Unofficially, to me, Krita now also multiclasses as a video game engine. Thank you for taking the time to read my article.