H-Score Methodology

<sub>Rules:
1. Benchmarks are omitted unless completed by the top 3 models.
2. Highest score in a benchmark used.
3. Score weighs all benchmarks evenly.
4. Scores are averaged to the nearest whole number.
5. Scores above 99 are not rounded.
6. Benchmarks will be added if ALL top 3 complete.
7. AI model must be available to public.</sub>

• Scoring Meta Data

*_{See Scoring Meta Data for inference details}

Inference Methodology

_{To compare performance to a reference model, we first calculate the absolute difference between their scores:}

_{This measures the raw gap between the two scores without considering which one is higher. Next, we normalize the difference by computing the average of both scores:}

_{This ensures a fair comparison across different benchmarks. Using these values, we calculate the percentage difference:}

_{Which standardizes the comparison. We then compute the average percentage difference across multiple benchmarks:}

_{This provides a balanced measure inferred model's overall performance gap relative to the reference models. Finally, we adjust the score by scaling down the reference score using the average percentage difference:}

_{This adjustment proportionally reduces the score, ensuring that model's inferred score accurately reflects its performance relative to the reference models. This step-by-step approach provides a consistent and standardized method for evaluating model’s effectiveness across multiple tasks.Inference scores are essential for obtaining a holistic score across benchmarks. AI companies often exclude benchmarks where their models underperform, introducing bias in comparisons. This inference score mitigates that bias by providing a more balanced and accurate assessment of overall performance.See this equation in action within the scoring meta data:}

• Scoring Meta Data

Current Benchmarks Scores

_{See Scoring Meta Data for full list of all 5 models.}

Benchmarks

Reporting Errors

_{If you believe that the ranking is missing data or is incorrect. Please direct message me on X.}

• Report

• Home
• Methodology
• RSI
• Journal
• About