Grab GPT-4o Vision Fine-Tuning for GrabMaps

Grab has utilized GPT-4o vision fine-tuning to automate the extraction of mapping data from street-level imagery, significantly increasing the accuracy of traffic sign localization and lane counting in Southeast Asia. This implementation reduces manual mapping efforts and operational costs while improving data reliability for Grab's 42 million monthly users and enterprise customers.

Automating Mapmaking with Vision Fine-Tuning

GrabMaps leverages GPT-4o vision fine-tuning to transform millions of street-level images—collected via 360-degree cameras on motorbike and pedestrian partner vehicles—into actionable mapping data. The model is used to localize speed limit signs, turn restrictions, places, and road geometries more accurately than previous methods.

To optimize the model, Grab conducted experiments focusing on matching speed limit signs to their corresponding roads. The process involved:

  • Dataset: Fine-tuning using 100 sample cases combining street-level imagery and map tiles.
  • Iteration: Adjusting hyperparameters over two rounds of fine-tuning.
  • Performance Gain: Baseline accuracy increased from 67% to 80%, representing a 13-percentage point improvement.

Technical Capabilities and Complex Scenario Handling

GPT-4o's vision capabilities allow GrabMaps to make context-aware decisions by cross-referencing street imagery with map tiles. This approach effectively handles complex geometries that previously required manual human intervention, specifically:

  • Elevated Roads: The model can distinguish between different road levels.
  • Occlusions: The model maintains accuracy even when signs or road features are are partially blocked.

According to Adrian Margin, Head of Data Science for Geo Mapping at Grab, "Fine-tuning GPT-4o with our data enabled us to handle complex geometries effectively, reducing manual interventions and operational costs."

Quantifiable Impact on Mapping Accuracy

The integration of vision fine-tuning has resulted in measurable improvements in data quality and operational efficiency:

  • Lane Count Accuracy: Increased by 20%.
  • Speed Limit Sign Localization: Improved by 13%.
  • Operational Costs: Reduced through the decrease in manual mapping efforts.
  • Data Trust: Higher accuracy in map outputs leads to increased trust in the data quality for internal operations and enterprise clients.

Future AI Initiatives at Grab

Beyond mapping, Grab is expanding its AI integration to improve accessibility and user support:

  • Conversational Voice Assistant: A multilingual tool currently in development to assist visually impaired and elderly users in navigating the app.
  • Advanced Support Chatbot: A system designed to handle complex inquiries by processing detailed standard operating procedures (SOPs) to provide empathetic and tailored responses.

Sources