Endor Labs Introduces New Evaluation Tool for Scoring AI Models
Ryan Daws is a senior editor at TechForge Media, bringing over ten years of expertise in creating engaging narratives and simplifying complex subjects. His insightful articles and interviews with industry experts have positioned him as a leading voice, recognized by organizations such as Onalytica. Publications under his guidance have received commendations from analyst firms like Forrester for their quality and performance. You can connect with him on X (@gadget_ry) or Mastodon (@gadgetry@techhub.social).
Endor Labs has launched a scoring system for AI models that evaluates them based on security, popularity, quality, and activity.
This new feature, known as ‘Endor Scores for AI Models,’ is designed to streamline the identification of the most secure open-source AI models available on Hugging Face. Hugging Face is a platform where users can share Large Language Models (LLMs), machine learning models, and other open-source AI models and datasets, providing clear scoring for easier selection.
The initiative comes as more developers are seeking out platforms such as Hugging Face for pre-built AI solutions, reminiscent of the earlier days of accessible open-source software (OSS). This advancement enhances AI governance by helping developers to “start clean” with AI models, a challenging objective that has yet to be fully realized.
Varun Badhwar, the Co-Founder and CEO of Endor Labs, expressed: “Our mission has always been to secure all dependencies of your code, and AI models represent the next significant challenge in this essential mission.
“Organizations are exploring AI models, whether to enhance specific applications or to create entire businesses based on AI. As security evolves, there’s a unique opportunity to establish robust measures from the outset, mitigating risks and avoiding elevated maintenance costs in the future.”
George Apostolopoulos, a Founding Engineer at Endor Labs, noted: “Currently, everyone is engaged in experimenting with AI models. Some teams are launching entirely new AI-driven businesses, while others are seeking to add a ‘powered by AI’ label to their offerings. One thing is clear: your developers are engaging with AI models.”
Nevertheless, this ease of use carries inherent risks. Apostolopoulos cautions that the current environment can be likened to “the wild west,” with individuals selecting models that meet their needs without regard for possible vulnerabilities.
Endor Labs views AI models as integral parts of the software supply chain.
As Apostolopoulos explains, “Our mission at Endor Labs is to ‘secure everything your code depends on.’” This viewpoint enables organizations to use the same risk assessment techniques for AI models that they utilize for other open-source components.
The tool developed by Endor focuses on various critical risk factors when evaluating AI models:
To address these challenges, Endor Labs’ assessment tool conducts 50 pre-defined checks on AI models hosted on Hugging Face. It produces an “Endor Score” that considers elements like the number of maintainers, corporate backing, frequency of releases, and known vulnerabilities.
Favorable elements in the evaluation system for AI models include the implementation of secure weight formats, the availability of licensing details, and robust download and engagement statistics. Conversely, adverse factors consist of insufficient documentation, missing performance metrics, and the utilization of insecure weight formats.
One notable attribute of Endor Scores is its accessibility for users. Developers are not required to remember specific model names; they can initiate their inquiries with broader questions such as “Which models can I use for sentiment classification?” or “What are the top models from Meta?” The tool then delivers straightforward scores ranking both the beneficial and detrimental features of each model, enabling developers to choose the most suitable options for their requirements.
“Your teams are being inquired about AI daily, and they will seek models that can help accelerate innovation,” emphasizes Apostolopoulos. “Assessing Open Source AI models with Endor Labs ensures that the models you’re implementing perform as expected and are safe for usage.”
(Photo by Element5 Digital)
Related article: China Telecom trains AI model with 1 trillion parameters on domestic chips
Interested in expanding your knowledge on AI and big data insights from industry experts? Attend the AI & Big Data Expo happening in Amsterdam, California, and London. This extensive event is held alongside other prominent conferences such as the Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.
Discover additional upcoming enterprise technology events and webinars powered by TechForge here.
Tags: ai, artificial intelligence, endor labs, evaluation, machine learning, model evaluation, models, scores
You need to be logged in in order to leave a comment.
Discover the pinnacle of WordPress auto blogging technology with AutomationTools.AI. Harnessing the power of cutting-edge AI algorithms, AutomationTools.AI emerges as the foremost solution for effortlessly curating content from RSS feeds directly to your WordPress platform. Say goodbye to manual content curation and hello to seamless automation, as this innovative tool streamlines the process, saving you time and effort. Stay ahead of the curve in content management and elevate your WordPress website with AutomationTools.AI—the ultimate choice for efficient, dynamic, and hassle-free auto blogging. Learn More
