| Course title | Natural language processing |
|---|---|
| Course code | KOL/ZPJ |
| Organizational form of instruction | Lecture + Seminar |
| Level of course | Bachelor |
| Year of study | 2 |
| Semester | Winter and summer |
| Number of ECTS credits | 5 |
| Language of instruction | Czech |
| Status of course | Compulsory-optional |
| Form of instruction | Face-to-face |
| Work placements | This is not an internship |
| Recommended optional programme components | None |
| Lecturer(s) |
|---|
|
| Course content |
|
1) Preprocessing of unstructured data 2) Processing of structured large data (XML, JSON) from kB to TB. 3) NLP frameworks: Spacy, Udpipe, FLAIR, SPARK and NLTK and basic NLP tasks: - sentence processing, - analysis of actor relationships based on dependency rules, - sentiment determination, - extraction of named entities. 4) Modeling and vectorization of text using Bag-of-Words: - advantages, disadvantages, classic modifications, use, - reduction using TF-IDF, SVD, PCA, - implementation methods, - calculations of text similarities. 5) Semantics - derivation of latent semantics based on PCA, SVD, MDS decompositions, - semantic embeddings Word2Vec, FastText, GloVe and their use, - use in text analysis. 6) Large Language Models (LLM) - introductory theory, reasons for existence, - convolutions, recurrent networks, - bidirectional transformers (BERT, etc.), - auto-regressive generative pre-trained transformers (GPT), - fine-tuning, - LLM limitations resulting from technology and solutions (tokenization, semantic shifts, pop-up windows, etc.), - generative LLMs as solvers (prompts, zero-shot, few-shot, outputs), - applications of generative LLMs (data ingestion, RAG, agents and swarms). 7) OCR - use of Tesseract, PyTesseract, EasyOCR and other tools, - implementation of OCR including preprocessing and postprocessing with language models. 8) Speech-to-Text, Text-to-Speech - currently available technologies and models Whisper, Seamless and others, - implementation of simple tasks.
|
| Learning activities and teaching methods |
| unspecified |
| Learning outcomes |
|
In this course, students will learn about modern methods of natural language processing: From preprocessing and working with large unstructured data, through classic NLP tasks, text vectorization and semantic embeddings, to current large language models (LLM). They will practically master working with selected NLP frameworks, the principles of BERT and GPT models from mathematical foundations to prompting and various applications. They will also master technologies such as OCR, Speech-to-Text and Text-to-Speech. Emphasis is placed on understanding the principles, possibilities and limitations of individual methods and their practical use in solving real-world problems.
1) Increase programming skills. 2) Acquire an understanding of typical tasks in practice and industry. 3) Acquiring tasks for research in linguistics. |
| Prerequisites |
|
1) Finished atleast 2nd semestr of programming in Python.
KOL/DATA2 ----- or ----- KOL/JMT2 ----- or ----- KOL/VMMT2 |
| Assessment methods and criteria |
|
unspecified
1) Completing tasks. 2) Active participation. |
| Recommended literature |
|
|
| Study plans that include the course |
| Faculty | Study plan (Version) | Category of Branch/Specialization | Recommended semester | |
|---|---|---|---|---|
| Faculty: Faculty of Arts | Study plan (Version): Lingvistics and Digital Humanities (2020) | Category: Philological sciences | 2 | Recommended year of study:2, Recommended semester: Winter |
| Faculty: Faculty of Arts | Study plan (Version): Lingvistics and Digital Humanities (2020) | Category: Philological sciences | 2 | Recommended year of study:2, Recommended semester: Winter |
| Faculty: Faculty of Arts | Study plan (Version): Lingvistics and Digital Humanities (2020) | Category: Philological sciences | 2 | Recommended year of study:2, Recommended semester: Winter |
| Faculty: Faculty of Arts | Study plan (Version): Lingvistics and Digital Humanities (2020) | Category: Philological sciences | 2 | Recommended year of study:2, Recommended semester: Winter |
| Faculty: Faculty of Arts | Study plan (Version): Lingvistics and Digital Humanities (2020) | Category: Philological sciences | 2 | Recommended year of study:2, Recommended semester: Winter |