Machine Learning of Natural Language Processing

This micro-credential has been carved out as a balanced selection of contemporary modules from the MA in Digital Text Analysis that targets industry professionals who are interested in learning about modern machine learning and natural language processing. With an emphasis on project and team work, we develop practical applications on textual data, as well as solutions for the many issues that remain open in the field.

Machine Learning

1. Introduction, history, linear regression, supervised and unsupervised learning, reinforcement learning, theoretical background, bias-variance

2. Methodology, sampling, cross-validation, stratification, preprocessing, normalization, tf-idf, pipelines. Working with sklearn.

3. Logistic regression, stochastic gradient descent

4. k-nn, clustering

5. Rule Induction, decision trees, decision forests

6. Naive Bayes, SVM, pipelines and tools for text problems in ML

7. Neural networks introduction.

8. Multilayer networks, feedforward perceptron, dropout, cetegorical cross-entropy; working with Pytorch

9. Methodology: batching, learning rate, optimizers

10. Convnets: images and text

11. LSTMs, recurrency, language models

12. Seq2seq, attention

Natural Language Processing

• Introduction to the course, NLP as part of Cognitive Artificial Intelligence: Perception, Action, Knowledge Representation, Problem Solving and Search, Machine Learning. Brief history of NLP.
• Introduction to Linguistics, morphology: Levenshtein, finite-state automata and transducers, probabilistic automata; Application: spelling error correction
• Rehash numpy, pandas, matplotlib. Intro Spacy, NLTK, Pattern/Grasp; exercises
• Syntax: Parsing, Shallow Parsing, statistical parsing, n-gram models, neural language models: Application: language models
• Deep Learning language models
• Semantics: word meaning: word sense disambiguation, distributional hypothesis, word embeddings; concept extraction, named entity recognition
• Exercises on making and using word embeddings
• Semantics: sentence meaning: semantic role labeling, argument structure, sentence embeddings
• Exercises on making and using sentence and document embeddings
• Discourse: cohesion, RST, coreference resolution
• Text categorization methodology: Applications: sentiment mining, profiling, topic modeling
• Text categorization pipeline in sklearn; Rehash ML methodology.
• Seq2seq neural models and their applications (information extraction, summarization, translation, simplification, ...)
• Exercises on seq2seq
• Guest lectures
• Student presentations

This micro-credential focuses on the following learning outcomes:

• The participant will have theoretical knowledge about the history and the main frameworks of Machine Learning.
• The participant will have theoretical knowledge of the main machine learning algorithms and paradigms and will study some of them in depth.
• The participant will be able to develop machine learning pipelines and set up their own machine learning experiments using Python modules.
• The participant will have theoretical and practical knowledge of machine learning using neural networks.
• The participant will understand the fundamental problems and approaches in automatic Natural Language Processing and know its history as a subfield within Linguistics and Artificial Intelligence.
• The participant will have insight into the basic algorithms developed within NLP for morphological, syntactic, semantic, and discourse processing.
• The participant will acquire hands-on experience with software for text categorization, language understanding, translation, and generation.

• All courses extensively rely on weekly, hands-on homework assignments, ensuring the acquisition of new, practical insights on a regular basis. The homework takes the form of engaging assignments on real-world datasets that challenge the students to apply the theoretical concept introduced during the interactive class sessions.
• The final evaluation of all three courses depends on project work, the goal and finality of which can be determined by the individual students, in close correspondence with the course teachers.
• An attractive feature of the evaluation of the NLP course (in the 2nd semester) is that students will participate in an ongoing shared task in the field.