On Wednesday, a San Francisco-based developer named Liu Liu released Draw things: AI generationa free application available on the App Store that allows iPhone owners to run the popular stable diffusion AI image generator. Write a description and the application will generate an image in several minutes. It’s a remarkable step in bringing image synthesis to a broader audience, with the added privacy of running it on your own hardware.
Introduced in August, Stable Diffusion (SD) is an AI image generator model that creates novel images from text descriptions (called “directions”). Usually people run SD through the commercial dream studio service, on a remote cloud machine with rented computing time, or locally on a PC using a custom open source implementation. When running locally, SD requires a fairly robust GPU to render images quickly, but some developers optimized the model to run on older GPUs with less VRAM (if you don’t mind waiting longer for results).
Along the same lines, Liu Liu has managed to optimize Stable Diffusion to run on the iPhone, a somewhat difficult process that the developer described in a blog post. “The main challenge is running the app on iPhone devices with 6 GiB of RAM,” writes Liu Liu. “6GiB sounds like a lot, but iOS will start killing your app if you use more than 2.8GiB on a 6GiB device and more than 2GiB on a 4GiB device.”
When you run Draw Things for the first time, the app downloads several necessary files, including the Stable Diffusion 1.4 model, to your iPhone. To use it, write a message at the top of the screen, then tap “Generate”. In between generating images, tap the number in the top center of the screen to randomize the seed, which is a number that partially guides the generation of the image.
On our iPhone 11 Pro, generating a 384×384 image took just over two minutes. It’s faster on an iPhone 14 Pro, according to Liu Liu, generating an image in about a minute. Either way, SD is computationally intensive. Over successive generations, our iPhone became noticeably warm to the touch.
It is worth noting that with Stable Diffusion, 384 × 384 images often produce relatively poor results with little detail because the SD creators trained the model on 512 × 512 images. When we try to generate a 512 × 512 image in our iPhone 11 Pro, we got a warning and continued anyway, but the app crashed and the screen went black.
In addition to regular image generation tasks, Draw Things also supports internal painting, allowing you to replace a part of an image with AI-generated images and load additional image synthesis models, such as the unauthorized “Modern Disney Broadcast“model (generating Disney-looking characters) and the anime”Waifu spread(we tested, and it is possible to generate NSFW material using the app, so be careful). The inclusion of those capabilities means that Draw Things might not stay on the App Store for long if it becomes popular, as the content it generates can violate Apple’s terms of service.
Whatever its destination, Draw Things feels like an important proof of concept, showing that Stable Diffusion can run locally on the iPhone, albeit slowly. If the app holds up, perhaps with some filtering or tweaking, Liu Liu describes the room for possible future optimizations that could speed things up: “I probably left 50% performance still on the table.”
This is not the first time that Stable Diffusion has run on an iPhone. In September, the developer matt waller ran Stable Diffusion locally on your iPhone XS, but did not provide an App Store app that would allow others to replicate the exploit. We are likely to see more local AI image synthesis in smartphones as devices continue to increase their computational power.