Interview, Fireside Chat, Other
Unlocking Creativity with Prompt Engineering
- The speaker expects a net shift in employment where job losses from new technologies are balanced by gains in new roles, though the immediate focus often remains on displacement.
- AI tools currently require a set of prompts to function effectively until they can close the loop on their own operation, with significant mastery estimated to require a couple of hundred hours despite the field being only about six months old.
- Within six months, tool capabilities, usage methods, and the overall landscape are predicted to change significantly, with new capabilities emerging in similarly short timeframes that were previously impossible.
- Early learnings in prompting are expected to evolve, with the field exploring where it is heading while facing a debate regarding whether text-to-image generation constitutes real artistry or merely discovers existing images in training data.
- Tools are currently less effective at specifying precise spatial relationships due to the general nature of real-life descriptions, and they struggle with physical rules and specific details like hands, though future iterations will likely focus aggressively on solving composition problems.
- Prompt length exhibits diminishing returns, though longer prompts allow for specific details such as camera angles, time periods, and artistry.
- Image-to-image capabilities, including the use of selfies and core image sets, are predicted to power the next generation of consumer interactions and create a new field of exploration.
- Users will increasingly cross over between tools, combining prompt generation with traditional editing software like Photoshop or mobile apps to achieve final results, while more models emerge enabling prosumers and organizations to build specialized versions via open-source efforts like Stable Diffusion.
- The industry is expected to move toward conversational interfaces that display multiple options in a multi-dimensional space to overcome the difficulty of verbal description, alongside better onboarding experiences to guide new prompters.
- Concepts similar to selling precise tuning, such as Instagram filters, will be applied to AI styles, potentially through training models on mood boards or specific visual concepts, though applying embedding tricks to specific artistic styles raises legal concerns.
- Users will seek to lock in specific styles into variables for future prompts to ensure consistency across content, with some startups already focusing on consistent looks for game assets even if this functionality is not yet fully integrated into foundation models.
- The industry anticipates a spectrum of careers similar to the film industry, with specialized niches for specific tasks like hair or hands, alongside a bimodal skill set where prompting is a fundamental tool for everyone and allows for "10x prompt engineers."
- A class of "secret prompting" professionals is expected to emerge to wrap prompts in additional layers similar to copywriters, adding unseen value to the average consumer's output.
- Evidence from communities and shared work will help determine if results are achieved through clever prompting or luck, while as tools improve, the artistry of the output may become less relevant than the resonance of the content, with memes remaining a dominant form of shared imagery.
- Future tools will likely continue to iterate on text-to-image models more aggressively to solve composition problems, and there will be a need to create methods for unleashing "inexplicable" or "undefinable" aesthetics that currently lack specific words.