COSMO: A large-scale e-commerce common sense knowledge generation and serving system at Amazon
COSMO is coming. A system designed to enhance e-commerce platforms by generating and utilizing common sense knowledge graphs. Traditional e-commerce knowledge graphs, while vast, often miss out on capturing user intentions, creating a disconnect between how users think, behave, and interact. COSMO aims to bridge this gap by mining user-centric common sense knowledge from extensive user behaviors and constructing industry-scale knowledge graphs to improve various online services.
COSMO utilizes a pipeline to collect high-quality seed knowledge assertions distilled from large language models (LLMs), which are further refined through critic classifiers trained on human-annotated data. The system faces challenges in aligning these machine-generated assertions with human preferences and filtering out noise. To address these, COSMO employs instruction tuning to fine-tune an efficient language model (COSMO-LM) for generating e-commerce common sense knowledge at scale.
The COSMO-LM has expanded Amazon's knowledge graph across 18 major categories, producing millions of high-quality knowledge assertions with only 30k annotated instructions. COSMO has been deployed in various Amazon search applications, including search relevance, session-based recommendation, and search navigation. The system has shown significant improvements in both offline and online A/B experiments, highlighting the potential of common sense knowledge extracted from instruction-finetuned large language models.
The document outlines the challenges faced by traditional methods in capturing user intentions and the innovative approach COSMO takes to generate, refine, and apply common sense knowledge at scale. It emphasizes the importance of aligning language models with human feedback and the efficiency gains from instruction tuning. The successful deployment of COSMO in real-world e-commerce tasks demonstrates its potential to significantly enhance user experience on e-commerce platforms.
The COSMO system targeted approximately 10% of Amazon's U.S. traffic in its online experiments. The tests demonstrated a significant 0.7% relative increase in product sales within this segment, translating to a substantial surge in annual revenue amounting to hundreds of millions of dollars. Additionally, there was an 8% increase in navigation engagement rate observed within the same traffic segment, highlighting improved customer interaction and satisfaction. This outcome suggests the potential for significant business growth if COSMO-LM is extended to encompass all traffic for navigation, potentially leading to a revenue increase in the billions​​.
The COSMO system, when applied to approximately 10% of Amazon's U.S. traffic, led to a notable 0.7% relative increase in product sales within this segment. This increase translated into hundreds of millions of dollars in annual revenue surge​​. Given this information, we can deduce that the 10% of traffic covered by the COSMO system contributed significantly to Amazon's overall revenue, indicating a substantial impact from the implementation of this technology.
The COSMO implementation involved several months of meticulously conducted online A/B tests by Amazon, targeting approximately 10% of Amazon's U.S. traffic​​. This suggests that the implementation and evaluation of COSMO have already taken place over an extended period, indicating that changes related to COSMO may already be impacting or could soon impact Amazon's search and recommendation systems.
What could this mean for existing and new sellers in the marketplace?
At the moment Amazon does not provide specific suggestions for sellers on Amazon to optimize for the changes brought by COSMO. The focus of the document is on the technical aspects and outcomes of implementing COSMO, such as the improvement in search relevance, recommendation systems, and user engagement. It discusses the architecture, deployment, and performance of COSMO but does not delve into strategies for sellers to adapt to these changes.
For sellers looking to optimize their listings in light of these advancements, it would be prudent to focus on enhancing the clarity and relevance of product information, ensuring accurate and comprehensive use of keywords related to user intentions, and improving customer engagement and feedback mechanisms. These strategies are generally beneficial for e-commerce optimization and might become even more critical as systems like COSMO become more prevalent in analyzing and understanding user behavior and intent.