Cost-Effective LLM Utilization for Machine Learning Tasks over Tabular Data

Classic machine learning (ML) models excel in modeling tabular datasets but lack broader world knowledge due to the absence of pre-training, an area where Large Language Models (LLMs) stand out. This paper presents an effective method that bridges the gap, leveraging LLMs to enrich tabular data to enhance the performance of classical ML models. Despite the previously limited success of direct LLM application to tabular tasks due to their high computational demands, our approach selectively enriches datasets with essential world knowledge, balancing performance improvement with cost-effectiveness. This work advances the capabilities of traditional ML models and opens new avenues for research at the convergence of classical ML and LLMs, marking the onset of a new era in cost-effective data enrichment.