Transformers, LLMs, RAG and Agents: From Theory to Production

Course Description

Purpose: This course addresses the growing demand for professionals who can effectively implement and deploy large language model applications. As organizations increasingly adopt AI technologies, there is a critical need for practitioners who understand not only the theoretical foundations but also the practical implementation of LLM-based systems.

Context: Large Language Models have revolutionized artificial intelligence, enabling unprecedented capabilities in natural language understanding, generation, and reasoning. This course bridges the gap between academic knowledge and industry application, covering the entire pipeline from model selection and fine-tuning to building production-ready RAG systems and autonomous agents.

Learning objectives:

• Understand and explain transformer architecture and attention mechanisms
• Implement fine-tuning workflows for pre-trained language models
• Design and deploy RAG systems using vector databases and embedding models
• Build custom MCP servers for tool integration with AI assistants
• Develop REACT agents capable of reasoning and autonomous action
• Evaluate trade-offs between different LLM deployment strategies (API vs local)
• Apply prompt engineering techniques for optimal model performance
• Integrate multiple AI components into cohesive application architectures

Agenda

The course will be self-paced 

Week 1: Transformers and Large Language Models

• Introduction to NLP and transformer architecture
• Hands-on with Hugging Face transformers library
• Fine-tuning pre-trained models
• Understanding LLM capabilities and emerging properties
• Prompt engineering fundamentals
• Working with LLM APIs and local deployment (Ollama, llama.cpp)
• Multi-modal LLMs overview

Week 2: RAG Assistants and Embeddings

• Vector embeddings and semantic search principles
• Embedding databases (Pinecone, Chroma, …)
• Retrieval Augmented Generation architecture
• Building simple assistants (Tool: https://lamb-project.org )
• RAG frameworks comparison vs custom implementations
• Case study the: Lamb Knowledge-base-server
• Optimization strategies for RAG systems

Week 3: Model Context Protocol (MCP)

• Introduction to Model Context Protocol
• Integrating MCP servers with development tools (VS Code, Cursor, Claude Desktop)
• MCP in software development workflows
• Building custom MCP servers
• Debugging and deployment best practices
• MCP server authentication and security

Week 4: REACT Agents and Advanced Integration

• Understanding the REACT (Reasoning and Acting) paradigm
• Building REACT agents from scratch
• Agent frameworks and orchestration
• Docker containerization for agents
• Integration with Google and OpenAI APIs
• Introduction to Agent-to-Agent (A2A) protocol

Prerequisites

• Proficient programming skills in Python (3+ years experience preferred)
• Understanding of machine learning fundamentals (supervised/unsupervised learning,
  model training)
• Familiarity with neural networks and deep learning concepts
• Experience with version control (Git) and command-line interfaces
• Basic knowledge of RESTful APIs and web services
• Recommended: Prior exposure to PyTorch or TensorFlow

Required readings or materials

1. “Attention Is All You Need” (Vaswani et al., 2017) – Foundational transformer
paperhttps://arxiv.org/abs/1706.03762
2. Hugging Face Transformers Documentation – Practical implementation
guidehttps://huggingface.co/docs/transformers/index
3. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (Lewis et
al., 2020)https://arxiv.org/abs/2005.11401
4. Model Context Protocol Documentation https://modelcontextprotocol.io/
5. “ReAct: Synergizing Reasoning and Acting in Language Models” (Yao et al.,
2022)https://arxiv.org/abs/2210.03629