This repository contains the code and scripts required to train a BERT model for sentiment analysis on financial news using GSB Slurm GPU node. The BERT model is trained on a Kaggle dataset of financial phrase bank. The project utilizes the transformers library along with PyTorch Lightning for streamlined model training and evaluation.
- Getting Started
- Repository Structure
- Training Script Details
- Usage
- Monitoring Training
- Output
- Authors
Sentences_AllAgree.txt: dataset of 2,264 financial news headlines labeled with a sentiment (Malo et al., 2014).finbert-job.sh: Slurm script for training the BERT model to be run on the Yens.finbert-train.py: Python script to train the model on Yen's GPU node.
The pre-check script check-cuda.py performs scan of the GPU configuration on job host.
The training script finbert-train.py performs the following:
- Defines a custom dataset to read in financial news texts and their labels.
- Utilizes the BERT model from the
transformerslibrary for sentiment analysis. - Trains the model using PyTorch Lightning's Trainer on Yen's GPU.
Modify the Slurm script to include your email address. Slurm will report useful metrics via email such as queue time, runtime, CPU and RAM utilization and will alert you if the job has failed.
Submit the Slurm script to initiate the model training on gpu partition on the Yens:
$ sbatch finbert-job.shMonitor the training progress by checking the Slurm queue for your username:
$ squeue -u $USER
$ sacct -j 3Instructions for monitoring GPU utilization and other training metrics.
login to compute nodes
$ nvtopAfter the training is complete, check the output file finBERT-train.out for training and evaluation metrics:
$ cat train-finbert.outputAuthors: Stanford GSB DARC Team (gsb_darcresearch@stanford.edu)
$ https://github.com/gsbdarc/yens-gpu-demo.git