Have LLMs Finally Mastered Geolocation? - bellingcat

Bellingcat analysed the geolocation performance of OpenAI , Google , Anthropic, Mistral and xAI's models.

We ran 500 tests, with 20 models each analysing the same set of 25 images.

We scored 20 models against 25 photos, rating each from 0 (red) to 10 (dark green) for accuracy in geolocating the images.

A photo of an alleyway in Singapore provoked a wide range of responses from the LLMs and Google Lens , with scores ranging from 3 (nearby country) to 10 (correct location) The LLMs can outperform Google Lens by focusing on small details in a photo to identify the exact location.

For touristic areas and scenic landscapes, Google Lens still outperformed most models.

In urban settings, LLMs excelled at cross-referencing subtle details.

ChatGPT was typically more confident than Gemini , often leading to better answers.

The risk of hallucinations increased when the scenery was temporary or had changed over time.