NLP: Supervised Sequence Prediction (Text to SQL)

project
May 2024

Translating Natural Language Instructions to SQL Queries (T5 Model)

  • This assignment focuses on supervised sequence prediction, specifically translating natural language instructions into SQL queries. Three different approaches will be explored: fine-tuning a pre-trained encoder-decoder transformer model (T5), training a similar model from scratch, and using diverse prompt engineering techniques with a large language model (LLM).
  • Tasks include:
  • 1. Fine-tuning a pre-trained T5 model.
  • 2. Training a T5 model from scratch.
  • 3. Using in-context learning with instruction-tuned LLMs (Gemma 1.1 2B and CodeGemma 7B).
  • Evaluation will be based on whether the generated SQL queries return the same database records as the ground-truth using the F1 metric. The assignment also includes a milestone task with an F1 score requirement of at least 0.5 on the test set by May 7.
  • Resources provided include a starter repository, data sets, and utility functions for evaluation. The assignment encourages experimentation with data processing, tokenization, and prompt design to improve model performance.
  • Submission involves code in a GitHub repository and a report following the provided template. The leaderboard ranks submissions based on F1 scores, with the final grade considering empirical performance, methodology, and code quality.
  • This assignment aims to provide experience in sequence prediction tasks, emphasizing empirical work and thorough analysis.
    • Sequence Prediction
    • SQL
    • Natural Language Processing
    • T5 Model
    Source Code