You can use markdown in job descriptions.

• add styles to the preview
• task list rendering doesn't work
• add stiles to public view

DALL·E Mini

Generate images from a text prompt

Our logo was generated with DALL·E mini using the prompt "logo of an armchair in the shape of an avocado".

How to use it?

There are several ways to use DALL·E mini to create your own images:

• use the official DALL·E Mini demo

• experiment with the pipeline step by step through our inference pipeline notebook

  

You can also use these great projects from the community:

• spin off your own app with DALL-E Playground repository (thanks Sahar)

• try DALL·E Flow project for generating, diffusion, upscaling in a Human-in-the-Loop workflow (thanks Han Xiao)

  

How does it work?

Refer to our report.

Contributing

Join the community on the LAION Discord. Any contribution is welcome, from reporting issues to proposing fixes/improvements or testing the model with cool prompts!

Development

Dependencies Installation

For inference only, use pip install git+https://github.com/borisdayma/dalle-mini.git.

For development, clone the repo and use pip install -e ".[dev]". Before making a PR, check style with make style.

Training of DALL·E mini

Use tools/train/train.py.

You can also adjust the sweep configuration file if you need to perform a hyperparameter search.

FAQ

Where to find the latest models?

Trained models are on 🤗 Model Hub:

• VQGAN-f16-16384 for encoding/decoding images
• DALL·E mini for generating images from a text prompt

Where does the logo come from?

The "armchair in the shape of an avocado" was used by OpenAI when releasing DALL·E to illustrate the model's capabilities. Having successful predictions on this prompt represents a big milestone to us.

Acknowledgements

• 🤗 Hugging Face for organizing the FLAX/JAX community week
• Google TPU Research Cloud (TRC) program for providing computing resources
• Weights & Biases for providing the infrastructure for experiment tracking and model management

Authors & Contributors

DALL·E mini was initially developed by:

• Boris Dayma
• Suraj Patil
• Pedro Cuenca
• Khalid Saifullah
• Tanishq Abraham
• Phúc Lê Khắc
• Luke Melas
• Ritobrata Ghosh

Many thanks to the people who helped make it better:

• the DALLE-Pytorch and EleutherAI communities for testing and exchanging cool ideas
• Rohan Anil for adding Distributed Shampoo optimizer and always giving great suggestions
• Phil Wang has provided a lot of cool implementations of transformer variants and gives interesting insights with x-transformers
• Katherine Crowson for super conditioning
• the Gradio team made an amazing UI for our app

Citing DALL·E mini

If you find DALL·E mini useful in your research or wish to refer, please use the following BibTeX entry.

@misc{Dayma_DALL·E_Mini_2021,
      author = {Dayma, Boris and Patil, Suraj and Cuenca, Pedro and Saifullah, Khalid and Abraham, Tanishq and Lê Khắc, Phúc and Melas, Luke and Ghosh, Ritobrata},
      doi = {10.5281/zenodo.5146400},
      month = {7},
      title = {DALL·E Mini},
      url = {https://github.com/borisdayma/dalle-mini},
      year = {2021}
}

References

Original DALL·E from "Zero-Shot Text-to-Image Generation" with image quantization from "Learning Transferable Visual Models From Natural Language Supervision".

Image encoder from "Taming Transformers for High-Resolution Image Synthesis".

Sequence to sequence model based on "BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension" with implementation of a few variants:

• "GLU Variants Improve Transformer"
• "Deepnet: Scaling Transformers to 1,000 Layers"
• "NormFormer: Improved Transformer Pretraining with Extra Normalization"
• "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows"
• "CogView: Mastering Text-to-Image Generation via Transformers"
• "Root Mean Square Layer Normalization"
• "Sinkformers: Transformers with Doubly Stochastic Attention"

Main optimizer (Distributed Shampoo) from "Scalable Second Order Optimization for Deep Learning".