Local text to image model comparaison: The ultimate test.

I selected 192 prompts to evaluate text-to-image model various capabilities and generated images for all the local models I was able to make work on my GX10 Spark.

For instance: Is the model good at text? At faces? At human anatomy? At respecting spatial composition, etc...? You just have to look at the images and have an idea by yourself.

You can see all the images here:

https://imagebench.ai/gallery?g=1_vbohinub2qwsahfzi_c11l7fi3.6wh838_lm

All the prompts are here: https://github.com/dh7/image-bench-ai

I also used some VLMs to evaluate the images. VLMs are not perfect, but they are good enough to understand how local models performed when compared to frontier APIs. Here are the results of this test: https://imagebench.ai/imagebench-v1

I hope you all find this useful, and I'm curious what I should test next on my GX10 Spark.

https://preview.redd.it/884996abvo8h1.png?width=2472&format=png&auto=webp&s=f5482c5391711a2186d5b4ff0bbd11d724a40aab