Building and Deploying LLMs

The training combines theoretical foundations, technical demonstrations, and independent exercises. The goal is for participants to develop the technical judgment needed to decide when to use prompting, RAG, fine-tuning, local models, external APIs, agents, or specialized infrastructure.

What You Will Learn

By the end of the course, participants will be able to:

• Understand how LLMs are built, adapted, and executed
• Analyze datasets from a technical, legal, and ethical perspective
• Distinguish between prompting, RAG, fine-tuning, agents, and local models
• Run open models locally using current tools
• Design applications with information retrieval, tools, and workflows
• Evaluate quality, safety, risks, and feasibility of a generative solution
• Apply basic criteria for governance, regulatory compliance, and responsible use

Agenda

Day 1: LLM fundamentals, pretraining, datasets, and licensing

• Introduction to LLMs, generative AI, and foundation models.
• Core concepts: tokens, Transformers, next-token prediction, scale, and compute.
• LLM lifecycle: pretraining, post-training, fine-tuning, evaluation, and deployment.
• Training data: public datasets, proprietary data, and synthetic data.
• Quality, cleaning, deduplication, bias, and dataset documentation.
• Licenses, copyright, personal data, and usage restrictions.
• Brief introduction to modern architectures such as Mixture of Experts.

Day 2: Post-training, fine-tuning, alignment, and evaluation

• Differences between pretraining, continued pretraining, instruction tuning, and fine-tuning.
• Supervised Fine-Tuning, LoRA, QLoRA, and PEFT.
• Alignment and preference-based RL methods: DPO, RLHF, GRPO.
• Dense models vs. Mixture of Experts: architecture, efficiency, and cost.
• Criteria for choosing between prompting, RAG, fine-tuning, or changing the model.
• LLM evaluation: benchmarks, golden datasets, rubrics, human and automated evaluation.
• Evaluation of hallucinations, safety, and model behavior.

Day 3: Local execution of LLMs: Ollama, llama.cpp, GGUF, quantization, and serving

• Motivations for running models locally: privacy, prototyping, cost, and control.
• Hardware requirements: CPU, GPU, memory, context size, and latency.
• Quantization and model formats, especially GGUF.
• Using tools such as Ollama and llama.cpp.
• Open models and local experimentation.
• Model serving and local APIs.
• Introduction to vLLM and comparison between local, cloud, external APIs, and HPC infrastructure.

Day 4: RAG, context engineering, tools, agents, and harnesses (Prof. Fabrício Carraro)

• Limitations of standalone LLMs.
• Prompt engineering vs. context engineering.
• RAG: document ingestion, chunking, embeddings, retrieval, reranking, and generation.
• When to use RAG versus fine-tuning.
• Tool calling and integration with external systems.
• Workflows vs. agents.
• Multi-agent flows.
• Current frameworks: LangChain, LangGraph, LlamaIndex, CrewAI, AutoGen.
• Schemas and Pydantic.
• Introduction to MCP as a standard for connecting models with tools and context.
• Risks: prompt injection, data leakage, tool misuse, and lack of traceability.
• Harnesses (Claude Code, Codex, Cursor, OpenCode)

Day 5 – Observability & ethical AI

• Principles of trustworthy AI and responsible use.
• Technical risks: hallucinations, bias, lack of robustness, and vendor dependency.
• Observability with Langfuse.
• Monitoring, red teaming, auditing, and criteria for moving from prototype to production.
• Security risks: prompt injection, data leakage, misuse of tools.
• GDPR, copyright, dataset licenses, and model licenses.