Follow-Up Projects

SEMINAR & MASTER-THESIS TOPICS

AI in Software Development · Three Frontiers to Join · SoSe 2026
Three frontiers you can actually join

CLUSTER 1

LLM Inference & Efficiency

Serve a million-token context without a terabyte of KV cache. Measure the GQA→MLA→sparse frontier.

builds on Lecture 5
CLUSTER 2

Evaluating Code & Agents

"95% on SWE-bench" measures a system, not a model. Take the benchmark apart and calibrate it.

builds on Lectures 8–10
CLUSTER 3

Transformers for Vision

The same attention you know — on image patches. DETR detection & frozen DINOv2 features.

same transformer, new modality

Each cluster has a 1-page deep-dive with ready-to-start topics. This deck is the trailer.

1

LLM Inference & the Efficiency Frontier

The lecture wrote the math. Nobody measured it.

Vanilla MHA
≈ 131 GB
GQA-8
≈ 16 GB
MLA + decoupled RoPE
≈ 2.3 GB
DeepSeek-V4 (sparse)
~10% of V3.2 @ 1M
KV cache per request · DeepSeek-V3-class model · 32K context (Lecture 5 numbers). Decoding is memory-bound: ~128 GB of cache reads per token.

The 2026 frontier (DeepSeek-V4; FlashMemory, arXiv:2606.09079) reframes the indexer as a retriever you can train without the backbone — runnable on one GPU precisely because you never touch the giant model.

Cluster 1 — the project menu

seminar / thesis ★
P1 · GQA vs MLA vs Sparse — a reproducible single-GPU KV-cache benchmark. Measured vs predicted, with error bars.
seminar / thesis
P2 · When Top-K Misses the Needle — when does sparse attention silently drop the one token that mattered? RULER + failure taxonomy.
ambitious
P3 · The Indexer Is a Retriever — a decoupled neural-memory indexer, trained without the backbone. The FlashMemory idea, tested.
seminar / thesis
P4 · The Efficient-Attention Zoo — a structured map from KV cache to 1M-token serving. One slide, the whole field.
seminar / thesis · opt
P5 · Speculative Decoding vs the Memory Wall — does drafting help when you are bandwidth-bound? Composition study.
2

Evaluating Code Generation & Agents

"95% on SWE-bench" is a measurement of a system

95%
=
Model
+
Scaffold
+
Grader
+
Data

Change any one of them — retries, test strength, leaked issue threads — and the number moves. Usually those conditions are undisclosed.

reported 12.47%
3.97%
← what a manual audit does once you remove leakage + weak tests (SWE-Bench+). On genuinely fresh issues: 0.55%.
Cluster 2 — the project menu

seminar / thesis
P1 · The Scaffold Tax — fix the model, vary only the harness: how many points are the scaffold? Agentless vs OpenHands.
seminar / thesis
P2 · Audit the Wins — re-read "resolved" instances by hand; replicate SWE-Bench+. Leakage vs weak tests.
seminar / thesis ★
P3 · Cost-per-Solved-Task — restore the suppressed cost axis; does the ranking reorder? Accuracy-per-euro Pareto.
seminar / thesis
P4 · Does context engineering help? — replicate the AGENTS.md "cost without gain" result. Success AND cost.
seminar / thesis
P5 · Benchmark-quality meta-eval — score today's coding benchmarks with a rubric. + one cheap diagnostic.
seminar / thesis
P6 · Faithfulness & hallucination — what should a faithfulness benchmark even measure? Cross-domain TRUST seed.
3

Transformers Beyond Text: Vision

The same transformer — pointed at a new modality

text → tokens
The
cat
sat
image → patches
→
Self-Attentionsame weights, same math
Detection became set prediction. DETR's object queries are learned slots.   DETR → RT-DETR: NMS-free, now real-time.
frozen Features became free. A DINOv2 backbone + a tiny trainable head — single-GPU, the "freeze the LLM, train a LoRA" move.
Cluster 3 — the project menu

seminar / thesis
P1 · DETR → Real-Time — map the lineage, then fine-tune RT-DETR vs YOLO on a domain set. Do DETRs finally beat YOLO?
seminar / thesis ★
P2 · Frozen DINOv2 + Light Head — how far do frozen features get on a domain task? Label-efficiency curve. The flagship.
seminar / thesis
P3 · Patch-Tokens Under the Microscope — ViT vs CNN + attention explainability. Attention you already know.
seminar / thesis
P4 · Transferring the LLM Lesson — token-merging efficiency for a ViT. The vision mirror of efficient attention.
ambitious
P5 · Few-Shot Domain Detector — frozen foundation features + a light detection head. Strong student only.
Pick your frontier

Cluster Pick if you like… Prereqs Safe flagship
1 · Inference GPUs, systems, the math of serving KV-Cache 5, PyTorch Reproducible KV-cache benchmark
2 · Eval & Agents measurement, skepticism, tooling Python, Docker, CLI Cost-per-solved-task Pareto
3 · Vision images, transfer learning, fast wins attention lectures, PyTorch Frozen DINOv2 + light head

Every topic here is single-GPU, public-data, and 6-month-real. The deep-dive for each cluster lists research questions, milestones, an honest scope, and a vetted reading list.

Which number will you
refuse to take on faith?

Teaser deck (NOT canvas_only -> Marp builds it to slides/*.html + .pdf). Companion deep-dives: prompts/lecture-ideas/seminar-thesis-clusters/ - cluster-1-llm-inference-and-efficiency.md - cluster-2-evaluating-code-generation-and-agents.md - cluster-3-transformers-for-vision.md Illustrations are scoped HTML/CSS (corporate palette), one <style scoped> per slide.