Don't trust the answers of a single AI

Yes, if you turn on web search before sending your question in the arena. Both models then receive the same extracts from pages found online, along with their addresses, and answer from them. The sources used appear under your message.

Without web search, models answer only from what they learned during training: they check nothing online, so recent or very specific facts may be missing.

Web search adds text to the request sent to the models. It therefore raises the number of tokens processed, and with it the energy shown in the summary.

The ranking rests on user votes and on nothing else. In each duel, two models answer blind and you pick the one you prefer.

Those duels feed a Bradley-Terry statistical model, which gives each model the level that best explains all the duels observed, expressed as an Elo score as in chess. Every score comes with a confidence interval: a model that has fought few duels holds a far less certain rank than one judged thousands of times.

By default the ranking is shown with style control, which cancels out the effect of presentation habits, answer length or formatting, so that only substance is compared. You can turn that control off and see the raw ranking.

It measures human preference on real uses, in European languages. It is not a knowledge test: a model leading here may be beaten on a technical exam.

Preference data is used to improve models during future training.

By blindly comparing the responses of two models, compar:IA users express their preferences, indicating which responses are most relevant. This preference data can be used to improve model alignment—that is, to train them to generate responses that are more in line with user expectations and preferences.

This is an iterative process, where the model gradually learns to generate better responses based on feedback from humans on the quality of the responses. By being exposed to preference data, models adjust their response style.

compar:IA uses the EcoLogits (GenAI Impact) methodology, which applies life cycle assessment (LCA) principles under the ISO 14044 standard. It currently covers two items: inference, that is using the model to answer, and the manufacture of the graphics cards (raw material extraction, production, transport).

The energy calculation starts from the model's active parameters and the number of tokens produced. A regression, fitted on the public measurements of the ML.ENERGY Leaderboard, gives the energy the graphics card spends per token. The share of the server hosting the model is added, and the total is multiplied by the data centre's PUE, which accounts for cooling and losses.

The reference hardware is that of data centres: an 80 GB NVIDIA H100 card, an 8-card server drawing 1.2 kW, a three-year lifespan and batches of 64 requests handled in parallel. The number of cards depends on the memory the model needs.

The CO₂ equivalent then follows from that electricity use, multiplied by the emission factor of the electricity mix of the country where the model runs.

For proprietary models, whose size is not published, EcoLogits works from a range: the figures shown are estimates, and are flagged as such.

It is worth noting that methods for assessing the environmental impact of AI are still being built.

Yes, the internationalization of compar:AI is underway. We are starting with an expansion to three pilot countries: Lithuania, Sweden, and Denmark. This first phase will allow us to test the approach and adapt the interface to different European linguistic and cultural contexts. Eventually, the circle may expand to more European languages based on feedback from these pilot countries. The objective is to gradually build a true European digital common for human evaluation of conversational AI, with collaborative governance that remains to be defined between the different participating countries.