What Big Tech Missed And How Startups Can Still Win

Predictions and Expectations:

• The speaker expects that if the company raises a billion and produces nothing for two years, it will be "very very hard to survive" due to external expectations.
• The speaker believes that Large Language Models (LLMs) will remain superior for language, mathematics, and programming, stating "works we will never beat i think the lm at that" for those domains.
• The speaker expects World Models to be "vastly superior" to LLMs for "high dimensional very noisy very long horizon problems."
• The speaker predicts that in five to ten years, there will be "helpful robots" and machines with common sense that will make "a big difference" to jobs, though "most jobs will still be here."
• The speaker expects that current robotics approaches (VLA) are "very bad hack" and "not accurate," implying they will not be the future for general-purpose robotics.
• The speaker believes that the current bottleneck for building foundational models is securing "data" and "gpus," even when money is available.
• The speaker is not sure if Mark Zuckerberg genuinely intended to hire 10,000 people for the 2015 concierge project, noting it could have been a tactic to reveal project risks.

Timelines and Milestones:

• The speaker plans for the company to communicate results internally but does not intend to share a specific timeline publicly "this year."
• The speaker expects that in the "short term" the primary application of World Models will be in robotics.
• The speaker anticipates that World Models will eventually be in production "everywhere."
• The speaker notes that the theoretical ideas behind LLMs (transformers) existed in "2018" but were not scaled until later.
• The speaker recalls the concept of World Models being discussed or existing in "five ten years" ago, citing JEPAs as an early example.

Technology and Product Direction:

• The company plans to build a "world model" by training on "video data," "audio," and robotic "touch" data rather than text, aiming to start from scratch "like a baby without any language."
• The speaker expects to use World Models to enable robots to operate in "open environment" rather than just narrow vertical tasks, aiming for safety and utility in homes or streets.
• The company intends to remain "upstream on the solution on the model" rather than just applying existing machine learning research, with the goal that "millions of people will use" the resulting product.
• The speaker expects the final trained World Model to be "more lightweight" with "less parameters" than LLMs, resulting in "less expensive" inference.
• The speaker plans to focus execution on securing "electricity GPUs data office" while the co-founder leads the "scientific vision" and "direction of the research."

Market and Industry Outlook:

• The speaker believes that the definition of AGI has shifted because early proponents have "changed the definition and stopped using the term," indicating the field is not yet achieving its original goals.
• The speaker expects that the ability to take "crazy things to take some risks" is a unique advantage startups have over large companies like Meta.
• The speaker notes that the market for foundational AI is shifting, with "most people agree that lms don't leave" general intelligence, creating an opening for alternative approaches.
• The speaker observes that securing GPU compute is "very very hard" even for companies with money, creating a significant barrier to entry.

Company Plans:

• The speaker plans to operate with offices in Paris, New York, Montreal, and Singapore, deliberately avoiding a headquarters in San Francisco to ensure team commitment.
• The company intends to hire "lots of people" in the US for the New York office, requiring employees to move and commit to the location.
• The speaker plans to cultivate a culture of spending money "as if it's yours" to counteract the "very tempting" nature of having raised one billion.
• The speaker plans to balance giving researchers "enough direction with enough freedom" to avoid them fleeing or running in random directions.
• The speaker intends to avoid the mistake of solving "a chatbot to do that does everything uh for everyone," advocating instead for solving a "very narrow problem" with a "very big vision."

Financial Guidance:

• The speaker states that raising "one billion in euros" (noting the 1.2 billion figure) creates a real cost in expectations rather than dilution.
• The speaker expects the cost of computing (GPUs) to be "very very expensive," requiring "billions of of euros" to build a foundational model.
• The speaker advises startups to "aim for billions, not millions," stating the difficulty depends on whether it is the first fundraising round, not the amount.
• The speaker notes that having raised seed money before made subsequent fundraising "easy" and that people eventually "beg you to give you money."

Risks and Caveats:

• The speaker warns that if the company raises a billion and "do nothing for two years," the external pressure will make survival "very very hard."
• The speaker identifies the risk that being "too early" will "kill" a company if the team's strength is go-to-market sales rather than engineering.
• The speaker cautions that trying to build a general chatbot without specific domain focus is "very hard to provide value as a very small team."
• The speaker warns that researchers "hate" being told what to do but will also "flee" if left alone without direction, creating a management balancing act.
• The speaker notes the risk that the market for foundational models is "very very expensive" and that securing the necessary hardware is difficult even with capital.

Confidence and Disagreement:

• The speaker is "super impressed" by the co-founder's background but acknowledges the co-founder has done "four companies" while the speaker has done "one."
• The speaker expresses confidence that World Models will eventually be "in production everywhere."
• The speaker is "not sure" if the Mark Zuckerberg meeting was a genuine offer or a strategy to expose risks in the project.
• The speaker disagrees with the notion that LLMs can solve high-dimensional, noisy, long-horizon problems, stating World Models will be "vastly superior."
• The speaker is confident that starting with a narrow problem but maintaining a "very very ambitious" vision is the correct approach.