1) Preprocessing of unstructured data 2) Processing of structured large data (XML, JSON) from kB to TB. 3) NLP frameworks: Spacy, Udpipe, FLAIR, SPARK and NLTK and basic NLP tasks: - sentence processing, - analysis of actor relationships based on dependency rules, - sentiment determination, - extraction of named entities. 4) Modeling and vectorization of text using Bag-of-Words: - advantages, disadvantages, classic modifications, use, - reduction using TF-IDF, SVD, PCA, - implementation methods, - calculations of text similarities. 5) Semantics - derivation of latent semantics based on PCA, SVD, MDS decompositions, - semantic embeddings Word2Vec, FastText, GloVe and their use, - use in text analysis. 6) Large Language Models (LLM) - introductory theory, reasons for existence, - convolutions, recurrent networks, - bidirectional transformers (BERT, etc.), - auto-regressive generative pre-trained transformers (GPT), - fine-tuning, - LLM limitations resulting from technology and solutions (tokenization, semantic shifts, pop-up windows, etc.), - generative LLMs as solvers (prompts, zero-shot, few-shot, outputs), - applications of generative LLMs (data ingestion, RAG, agents and swarms). 7) OCR - use of Tesseract, PyTesseract, EasyOCR and other tools, - implementation of OCR including preprocessing and postprocessing with language models. 8) Speech-to-Text, Text-to-Speech - currently available technologies and models Whisper, Seamless and others, - implementation of simple tasks.
|
In this course, students will learn about modern methods of natural language processing: From preprocessing and working with large unstructured data, through classic NLP tasks, text vectorization and semantic embeddings, to current large language models (LLM). They will practically master working with selected NLP frameworks, the principles of BERT and GPT models from mathematical foundations to prompting and various applications. They will also master technologies such as OCR, Speech-to-Text and Text-to-Speech. Emphasis is placed on understanding the principles, possibilities and limitations of individual methods and their practical use in solving real-world problems.
1) Increase programming skills. 2) Acquire an understanding of typical tasks in practice and industry. 3) Acquiring tasks for research in linguistics.
|