In this chapter, we train a neural machine translation (NMT) model by using IWSLT’14 English to German translation dataset. For the actual implementation of an NMT model, use the off-the-shelf toolkits such as fairseq, Hugging Face Transformers and OpenNMT-py.
90. Data Preprocessing
Download the dataset and obtain training, validation and test data by running this script.
91. Training the machine translation model
Use the dataset from the problem 90 and train the machine translation model. The choice of the model architecture is arbitrary (e.g., LSTM-based model, Transformer-based model).
92. Translating a text
Use the trained model from the problem 91 and implement the program that translates a given English sentence to a German sentence.
93. BLEU score
Use the trained model from the problem 91 and compute the BLEU score on the test set.
94. Beam search
Use the beam search when translating a given sentence with the model from the problem 91. Change the beam width from 1 to 100. Then, plot the change in BLEU score on the graph.
Use the subword-based segmentation instead of the word-based segmentation. Repeat the problem 91-94 with the new segmentation.
96. Plotting the learning curve
Use tools like TensorBoard and visualize how the training of the machine translation model proceeds. Specifically, visualize the metrics such as the loss and the BLEU score on the training data, and the loss and the BLEU score on the validation data.
97. Hyper-parameter search
Change the architecture and/or hyper-parameter of the machine translation model and search for the model that achieves the highest performance on the validation data.
98. Domain Adaptation
99. Translation Server
Build a demo system, in which the user can translate arbitrary text on the Web browser.