Google Introduces Gemini 1.5: An ‘Experimental’ Feature with 1M Token Context
Ryan is a senior editor at TechForge Media with over a decade of experience covering the latest technology and interviewing leading industry figures. He can often be sighted at tech conferences with a strong coffee in one hand and a laptop in the other. If it’s geeky, he’s probably into it. Find him on Twitter (@Gadget_Ry) or Mastodon (@gadgetry@techhub.social)
Google has unveiled its latest AI model, Gemini 1.5, which features what the company calls an “experimental” one million token context window.
The new capability allows Gemini 1.5 to process extremely long text passages – up to one million characters – to understand context and meaning. This dwarfs previous AI systems like Claude 2.1 and GPT-4 Turbo, which max out at 200,000 and 128,000 tokens respectively:
“Gemini 1.5 Pro achieves near-perfect recall on long-context retrieval tasks across modalities, improves the state-of-the-art in long-document QA, long-video QA and long-context ASR, and matches or surpasses Gemini 1.0 Ultra’s state-of-the-art performance across a broad set of benchmarks,” said Google researchers in a technical paper (PDF).
The latest model from Google gains its effectiveness from its avant-garde Mixture-of-Experts (MoE) design.
“Instead of operating as a big, single neural network like traditional Transformers, MoE models split into smaller ‘expert’ neural networks,” said Demis Hassabis, the honcho of Google DeepMind.
“The MoE models are designed to activate only the pertinent expert pathways in its neural network, based on the kind of input provided. This specialization intensifies the efficiency of the model greatly.”
The prowess of the 1M token context window was highlighted by Google using Gemini 1.5. The model was able to consume the entire 326,914-token Apollo 11 flight manuscript and then accurately responded to precise queries about it. It also concisely highlighted crucial details from a 684,000-token silent movie as instructed.
Google is initially providing developers and enterprises free access to a limited Gemini 1.5 preview with a one million token context window. A 128,000 token general release for the public will come later, along with pricing details.
In December, we launched Gemini 1.0 Pro. Today, we’re introducing Gemini 1.5 Pro! 🚀
This next-gen model uses a Mixture-of-Experts (MoE) approach for more efficient training & higher-quality responses. Gemini 1.5 Pro, our mid-sized model, will soon come standard with a… pic.twitter.com/m2BNufHd8C
For now, the one million token capability remains experimental. But if it lives up to its early promise, Gemini 1.5 could set a new standard for AI’s ability to understand complex, real-world text.
Developers interested in testing Gemini 1.5 Pro can sign up in AI Studio. Google says that enterprise customers can reach out to their Vertex AI account team.
(Image Credit: Google)
See also: Amazon trains 980M parameter LLM with ’emergent abilities’
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
Tags: ai, ai model, ai studio, artificial intelligence, deepmind, developers, enterprise, gemini, gemini 1.5, Google, google gemini, mixture-of-experts, moe, neural networks, vertex ai
You must be logged in to post a comment.
Discover the pinnacle of WordPress auto blogging technology with AutomationTools.AI. Harnessing the power of cutting-edge AI algorithms, AutomationTools.AI emerges as the foremost solution for effortlessly curating content from RSS feeds directly to your WordPress platform. Say goodbye to manual content curation and hello to seamless automation, as this innovative tool streamlines the process, saving you time and effort. Stay ahead of the curve in content management and elevate your WordPress website with AutomationTools.AI—the ultimate choice for efficient, dynamic, and hassle-free auto blogging. Learn More
