Lecture slides
Lecture 1 – Introduction and history
Lecture 2 – Using AI for scientific writing and development
Lecture 3 – Responsible AI and EU legal context
Lecture 4 – Resources and evaluation
Exercises
Transformer visualizer [5, 6, 7]
RAG comparison [12]
EU AI Act compliance [3, 4, 9]
LLM-as-a-Judge [8, 13, 14]
Literature
[1] S. Russell and P. Norvig, Artificial Intelligence – A Modern Approach (4th Edition)
[2] R. Bommasani et al., On the opportunities and risks of foundation models, arXiv:2108.07258v3
[3] The EU Artificial Intelligence Act
[4] E. Bender et al., On the dangers of stochastic parrots: Can language models be too big?, ACM (2021)
[5] A. Vaswani et al., Attension is all you need, arXiv:1706.03762v7
[6] J. Delvin et al., BERT: Pre-training of deep bidirectional transformers for language understanding, NAACL (2019)
[7] K. Clark et al., What does BERT look at? An analysis of BERTs attention, arXiv:1906.04341v1
[8] K. Papineni etal., BLEU: A method for automatic evaluation of machine translation, ACL (2002)
[9] Yuntao Bai etal., Constitutional AI: Harmlessness from AI feedback, arXiv:2212.08073v1
[10] T. Brown et al., Language models are few-shot learners, arXiv:2005.14165
[11] J. Kaplan et al., Scaling laws for neural language models, arXiv:2001.08361
[12] P. Lewis et al., Retrieval-Augmented Generation for knowledge-intensive NLP tasks, arXiv:2005.11401
[13] H. Naveed et al., A comprehensive overview of Large Language Models, arXiv:2307.06435
[14] L. Ouyang et al., Training language models to follow instructions with human feedback, arXiv:2203.02155
[15] C. Raffael et al., Exploring the limits of transfer learning with a unified text-to-text transformer, JMLR (2020)
[16] H. Touvron et al., LLaMA: Open and efficient foundation language models, arXiv:2302.13971
Working material
The prompt: Summarize this article. Include method, main results, conclusion, impact on the field. Use bullet points to structure output. It sould be scientifically sound. State uncertainty where evidences are insufficient.