We are announcing three new members to our Board of Directors as the first step towards our commitment to expansion: dr. Sue Desmond-Hellmann, former executive director of the Bill and Melinda Gates Foundation, Nicole Seligman, former EVP and General Counsel at Sony Corporation and Fiji Simo, CEO and President of Instacart. Additionally, Sam AltmanCEO, will…

Working on cutting-edge AI is unfortunately expensive. For example, in addition to DeepMind, Google also has Google Brain, Research and Cloud. And TensorFlow, TPUs, and they own about a third of all research (in fact, they hold their own AI conferences). I also strongly doubt that computing power will be necessary (and perhaps even sufficient)…

The Internet contains a huge amount of publicly available videos from which we can learn. You can watch a person create a beautiful presentation, a digital artist draw a beautiful sunset, and a Minecraft player build an intricate house. However, these videos only provide a recording what happened but not exactly how this is achieved,…

We noticed that our internal DALL·E 2 predecessors would sometimes reproduce the training images verbatim. This behavior was undesirable, since we would like DALL·E 2 to create original, unique images by default, not just “merge” parts of existing images. In addition, verbatim reproduction of training images may raise legal issues regarding copyright infringement, ownership, and…

Codex, a large language model (LLM) trained on various codebases, exceeds the prior art in its ability to synthesize and generate code. While Codex provides many advantages, models that can generate code on such a scale have significant limitations, compatibility issues, potential for abuse, and the potential to increase the rate of progress in technical…

We show that autoregressive language models can learn to fill in text after we apply a direct transformation to the dataset, which simply shifts the range of text from the middle of the document to its end. Although this data augmentation has attracted considerable interest in recent years, we provide extensive evidence that training models…

Other existing approaches often use smaller, more closely matched datasets for audio-text training,(^reference-1) (^reference-2)(^reference-3) or use a wide but unsupervised audio pre-exercise.(^reference-4)(^reference-5)(^reference-6) Because Whisper was trained on a large and diverse dataset and wasn't fine-tuned to any particular one, it doesn't outperform models that specialize in the performance of LibriSpeech, a well-known competitive benchmark in…

In reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences. Since the reward model is an imperfect substitute, over-optimizing its value may compromise the ground-truth performance, according to Goodhart's law. This effect is often observed but not carefully measured due to the cost of collecting…

While recent work on text-driven 3D object generation has shown promising results, state-of-the-art methods typically require multiple GPU hours to produce a single pattern. This is in stark contrast to state-of-the-art generative imaging models, which produce samples in seconds or minutes. In this paper, we explore an alternative method for generating 3D objects that produces…

As generative language models improve, they open up new possibilities in fields as diverse as health, law, education, and science. But as with any new technology, it's worth considering how it can be misused. Against the background of repetitive online influence operations—disguised or tricky efforts to influence the opinions of the target audience – the…