Interview
The inside story of how ChatGPT was built – OpenAI cofounder John Schulman
- The next model iteration is planned to be conversational, designed to facilitate user follow-up questions and enable the model to ask clarifying questions.
- The model demonstrated strong coding assistance capabilities, which remained a key area of excitement despite the team later de-emphasizing browsing features due to the internal knowledge base's robustness.
- Focus shifted following GPT-4's training completion in August, affecting prior plans for a public release and contributing to a temporary market shift away from chat interfaces after the release of instruction-following GPT-4 fine-tunes.
- Chat models were developed to offer a more intuitive and easier user experience compared to previous instruction models, addressing the latter's poorly defined tasks and labeling confusion.
- A clearer conceptual understanding of helpful robot behavior facilitated the development of a coherent personality and robust, sensible behavior in chat models.
- Replicating similar results via external fine-tuning APIs would likely require several iterations or iterative supervised fine-tuning with human-edited outputs, as a single iteration is insufficient for models to perfectly fit human-generated data.
- Prior instruction models showed competence in creative writing and poetry but lacked proficiency regarding factuality and knowledge of limitations.