Deep Learning for Natural Language Processing (131307379A)

Overview

In recent years, natural language processing (NLP) research has undergone a massive transformation. The emergence of large language models (LLMs) has dramatically improved the ability to generate and understand text, revolutionizing various application domains such as translation, question answering, and summarization. In 2024-2025, multimodal LLMs like GPT-5 and Gemini 2.5 Pro that can simultaneously process text, images, and audio have emerged, further expanding the scope of applications. Particularly noteworthy is the emergence of new architectures beyond Transformer. For example, Mamba, a state space model (SSM), can efficiently process up to millions of tokens with linear O(n) complexity, while RWKV can process conversational messages at 10x or more lower cost than existing methods in real-time.

This course reflects these latest developments to provide hands-on deep learning-based NLP techniques. Students first learn core tool utilization methods such as PyTorch and Hugging Face usage, then directly experience fine-tuning of Transformer-based models and latest SSM architectures, prompt engineering, retrieval-augmented generation (RAG), reinforcement learning from human feedback (RLHF), and agent framework implementation. Additionally, we cover latest parameter-efficient fine-tuning (PEFT) techniques (WaveFT, DoRA, VB-LoRA, etc.) and advanced RAG architectures (HippoRAG, GraphRAG), and practice cutting-edge concepts such as multimodal LLMs and ultra-long context processing. Finally, through team projects, students integrate learned content to implement complete models and applications that solve real problems.

This course is designed for third-year undergraduate level and assumes completion of prerequisite course Language Models and Natural Language Processing (131107967A). Through team projects, students challenge real problem-solving using Korean corpora, and in the final project phase, we provide opportunities to work with industry datasets and receive feedback from industry experts, considering industry-academia collaboration.

Learning Objectives

• Understand the role and limitations of large language models in modern NLP and utilize related tools such as PyTorch and Hugging Face.

• Understand the principles and trade-offs of State Space Models (e.g., Mamba, RWKV) along with latest architectures.

• Apply fine-tuning to pre-trained models or latest parameter-efficient fine-tuning methods like WaveFT, DoRA, VB-LoRA.

• Learn methods to systematically optimize prompts using prompt engineering techniques and DSPy framework.

• Understand the evolution of evaluation metrics (e.g., G-Eval, LiveCodeBench, etc.) and the importance of human evaluation, and learn latest alternatives to RLHF such as DPO (Direct Preference Optimization).

• Design and implement advanced RAG (Retrieval-Augmented Generation) architectures like HippoRAG, GraphRAG and hybrid search strategies.

• Understand AI regulatory frameworks like EU AI Act and acquire methodologies for implementing responsible AI systems.

• Track latest research trends to discuss multimodal LLMs, small language models (SLM), state space models (SSM), multi-agent systems, mixture of experts (MoE), and other diverse latest technologies.

• Understand the characteristics and challenges of Korean NLP and develop application capabilities through hands-on practice using Korean corpora.

• Strengthen collaboration and practical problem-solving capabilities through team projects and gain project experience connected to industry.

Course Schedule

Week	Main Topics and Keywords	Key Hands-on/Assignments
1	Transformer and Next-Generation Architectures • Self-Attention Mechanism and Limitations • Mamba (SSM), RWKV, Jamba	Transformer Component Implementation Mamba vs Transformer Performance Comparison Architecture Complexity Analysis
2	PyTorch 2.x and Latest Deep Learning Frameworks • torch.compile Compiler Revolution • FlashAttention-3 Hardware Acceleration • AI Agent Frameworks	torch.compile Performance Optimization FlashAttention-3 Implementation AI Agent Framework Comparison
3	Modern PEFT Techniques for Efficient Fine-tuning • LoRA, DoRA, QLoRA • Advanced PEFT Techniques	PEFT Method Comparison Experiment LoRA/DoRA/QLoRA Performance Evaluation Memory Efficiency Analysis
4	Advanced Prompt Techniques and Optimization • Prompt Engineering Fundamentals • Self-Consistency, Tree of Thoughts • DSPy Framework	DSPy-based Automatic Prompt Optimization Self-Consistency Implementation Tree of Thoughts Problem Solving
5	LLM Evaluation Paradigms and Benchmarks • Evaluation Paradigm Evolution • LLM-as-a-Judge (GPTScore, G-Eval, FLASK) • Specialized and Domain-specific Benchmarks	G-Eval Implementation Benchmark Comparison Experiment Evaluation Bias Analysis
6	Multimodal NLP Advancements • Vision-Language Models (LLaVA, MiniGPT-4, Qwen-2.5-Omni) • Visual Reasoning (QVQ-Max) • Speech Integration	Multimodal QA Application Development Vision-Language Model Comparison End-to-end Multimodal System
7	Ultra-Long Context Processing and Efficient Inference • Context Window Revolution (1M+ tokens) • Attention Mechanism Optimization • LongRoPE and RAG Integration	FlashAttention-3 Integration Long Context Processing Comparison Performance Analysis
8	Core Review and Latest Trends • Architecture Review • Latest Model Trends (GPT-5, Gemini 2.5 Pro, Claude 4.1) • Industry Applications	Comprehensive Review Model Comparison Industry Case Analysis
9	Advanced RAG Systems – HippoRAG, GraphRAG, Hybrid Search Strategies	Assignment 3: Building Korean Enterprise Search System based on GraphRAG
10	Innovation in Alignment Techniques – DPO, Constitutional AI, Process Reward Models	Comparison Practice between DPO and Existing RLHF Techniques
11	Production Agent Systems – CrewAI, Mirascope, Type-Safety Development	Multi-agent Orchestration Implementation
12	AI Regulation and Responsible AI – EU AI Act, Differential Privacy, Federated Learning	Assignment for Designing Regulation-Compliant AI Systems
13	Ontology and AI – Modeling Reality and Operating it with AI • Data Science to Decision Science • Semantic Ontology, GraphRAG • Kinetic Ontology, Closed-Loop Systems	Semantic Ontology Modeling GraphRAG Implementation Closed-Loop Simulation
14	Final Project Development and MLOps	Team Prototype Implementation and Feedback Sessions (Industry Mentor Participation)
15	Final Project Presentations and Comprehensive Evaluation	Team Presentations, Course Content Summary and Future Prospects Discussion

Table of Contents

Lecture Notes

• Week 1: Transformer and Next-Generation Architectures
• Week 2: PyTorch 2.x and Latest Deep Learning Frameworks
• Week 3: Efficient Fine-Tuning with Modern PEFT Techniques
• Week 4: Advanced Prompting Techniques and Optimization
• Week 5: LLM Evaluation Paradigms and Benchmarks
• Week 6: Advances in Multimodal NLP
• Week 7: Ultra-Long Context Processing and Efficient Inference
• Week 8: Core Review and Latest Trends
• Week 9: Advanced RAG Architectures
• Week 10: Revolutionary Alignment Techniques
• Week 11: Production Agent Systems
• Week 12: AI Regulation and Responsible AI
• Week 13: Ontology and AI
• Week 14: The 2025 NLP Landscape

Workshops

• LLM From Scratch Workshop
• Week 1 Workshop: LLM Overview and Development Environment Setup

Projects

• Team Project Guidelines

About

• Syllabus
• Who made this book?