#303: Meta Aims to Build the World’s Largest AI Supercomputer, & More

Last week, Meta released details of AI Research SuperCluster (RSC), a new supercomputer powered by more than 6,000 Nvidia A100 datacenter graphic processing units (GPUs), that will boost the performance of its artificial intelligence (AI) clusters by 20x. For perspective, the phase one rollout of RSC is comparable in size to Tesla’s in-house Nvidia supercomputer. In phase two, with 16,000 GPUs, Meta aims to build the world’s largest supercomputer.

Focused on privacy, Meta researchers will leverage the additional computing power to improve training models that identify harmful content on its platforms, enable real-time video translation, and power advanced robotics. To make these meaningful advances, Meta plans to train models with trillions of parameters, an order of magnitude higher than GPT-3’s 175 billion.

Last week, the U.S. Securities and Exchange Commission proposed amendments to Regulation ATS (“alternative trading systems”) in the Exchange Act originally proposed in September 2020. Addressed by Gabriel Shapiro, the proposal would expand the definition of a “securities exchange” to include crypto decentralized finance (DeFi) platforms and automated market-making (AMM) protocols, subjecting both to SEC oversight. The public comment period will run for 30 days.

The proposed definition of a securities exchange would extend to “communication protocols,” potentially including blockchain search engines like Etherscan. If so, the SEC could oversee and regulate the open-source data in public blockchain systems. Though we are not certain, the SEC could be proposing an expansion in its regulatory oversight from marketplaces that auction securities between actual buyers and sellers to the protocols communicating trading interests of potential buyers and sellers.

In our view, if it were to apply to blockchain search engines, this amendment could hinder innovation in the US financial services sector, potentially incentivizing entrepreneurs and businesses to migrate operations offshore. We also would question the amendment’s constitutionality. Would SEC oversight of communication protocols that enable the exchange of publicly available information about buyer and seller interests violate the First Amendment?

While large computational models enable AI systems to generate language with impressive accuracy, sometimes they generate text that fails to address the given prompt or is biased and otherwise incorrect. To address those shortcomings, OpenAI developed a new language model––InstructGPT––that uses “reinforcement learning from human feedback” (RLHF) to improve the usefulness and safety of model outputs. In RLHF, human labelers “coach” the model with examples of desired behavior and scoring outputs, correcting and teaching it iteratively to achieve better results.

According to OpenAI, InstructGPT outperforms its predecessor language mode, GPT-3, in various tests: RealToxicity, which measures the toxicity of outputs, and TruthfulQA, which measures for falsehoods. InstructGPT is now the default language model powering the OpenAI API.

Last week, ARK released Big Ideas 2022 which highlighted that autonomous ride-hail could increase global GDP by $26 trillion, potentially becoming the most impactful innovation in history by 2030. Previously, ARK estimated that by increasing the utilization of vehicles, an autonomous ride-hail operator operating at scale could offer transportation services for as little as $0.25 per mile. ARK’s recent research on autonomous ride-hail analyzed the potential pricing tiers that might evolve and now estimates $4 per mile at the upper end and $0.25 per mile at the lower end. All in, our estimate for the total addressable market for autonomous ride-hail is $11 trillion.

ARK expects those price points to generate explosive growth in global ride-hail consumption by 2030. Our research aligns with Elon Musk’s suggestion during Tesla’s earnings call last Thursday that future ride-hail could cost as little as $0.20 per mile and that autonomous capability could be “the most profound software upgrade in history.”