Neural Machine translation with a Seq2seq model

Welcome to our Seq2seq repository! The goal of the project is to implement a NMT model to perform translation from French to English. To do so, we relied on a PyTorch tutorial written by Sean Robertson : NLP From Scratch: Translation with a Sequence to Sequence Network and Attention.

The project is an academic project from CentraleSupélec realized as part of the NLP class.

Getting started

pip install -r requirements.txt 
nltk.download('punkt')
jupyter notebook seq2seq_notebook.ipynb 

We relied on the model provided in the PyTorch tutorial. We trained it on datasets recording Ted talks and their translations. In addition to this, we implemented beam search with length penalty. Used at inference time, it is coded in a script called beam_search.py.

Project structure

The structure of our project is as follow :

• seq2seq_notebook.ipynb 
• beam_search.py
+ ted_data
 + train 
   • IWSLT14.TED.dev2010.en-fr.en.xml 
   • IWSLT14.TED.dev2010.en-fr.fr.xml
   • IWSLT14.TED.tst2010.en-fr.en.xml
   • IWSLT14.TED.tst2010.en-fr.fr.xml
  + test
   • IWSLT14.TED.tst2011.en-fr.fr.xml
   • IWSLT14.TED.tst2011.en-fr.en.xml 
   • IWSLT14.TED.tst2012.en-fr.en.xml
   • IWSLT14.TED.tst2012.en-fr.fr.xml

Results

	BLEU scores experiment 1
Beam size	Training set	Test set
1	7,28	0,30
2	8,36	0,37
5	9,36	0,31
10	9,35	0,37
50	9,37	0,36

	BLEU scores, experiment 2
Beam size	Training set	Test set
1	7,15	0,29
2	8,09	0,30
5	8,18	0,40