01 Lecture 1 Quiz: AI in Software Development

AlexNet won the 2012 ImageNet challenge with a top-5 error rate of 15.3% — nearly half the error of the runner-up (26.2%). This proved that deep convolutional neural networks could far outperform traditional computer vision approaches, kickstarting the deep learning revolution.

ResNet introduced skip connections (residual connections), which allow information to bypass layers by adding the input of a block directly to its output. This solved the vanishing gradient problem and enabled training of networks with 152+ layers.

The transformer replaced sequential processing (RNNs) with self-attention mechanisms that process all tokens in parallel. This enabled massive parallelization during training and better capture of long-range dependencies, becoming the foundation for virtually all modern LLMs.

BERT (Google, 2018) reads text in both directions, making it excellent at understanding tasks like classification and question answering. GPT (OpenAI, 2018) reads left-to-right, making it a generative model that produces new text token by token. For code generation, GPT’s left-to-right approach is well-suited.

GPT-3 demonstrated emergent abilities — capabilities that appeared as a consequence of scale rather than explicit training. Most notably, few-shot prompting allowed GPT-3 to perform tasks by showing it just a few examples, without any fine-tuning. This was a qualitative leap from previous models.

Codex (2021) was created by fine-tuning GPT-3 on billions of lines of public code from GitHub. It became the engine behind the first version of GitHub Copilot, enabling developers to type a function signature and have AI complete the implementation — marking the beginning of AI-assisted coding.

ChatGPT was built on GPT-3.5 (GPT-4 was months away) and was not the first chatbot. But powered by RLHF from InstructGPT, it was the first AI that felt conversational enough for mainstream adoption — reaching 1 million users in 5 days and 100 million in 2 months, the fastest-growing consumer application in history.

Before Llama 2, frontier models were proprietary. Llama 2 with open weights meant anyone could download, fine-tune, host, and modify a competitive language model for free. This spawned an enormous open-source ecosystem and challenged the assumption that only well-funded companies could provide state-of-the-art AI.

The evolution: (1) Chat-based (2022-2023): copy-paste from browser. (2) IDE-integrated (2023-2024): inline completions in editors. (3) Agent mode (2024-2025): AI reads files, runs commands, creates PRs. (4) Autonomous workflows (2025+): AI handles full tasks end-to-end with MCP tool connectivity.

Stack Overflow traffic dropped approximately 50%, with monthly visits falling from roughly 1.5 billion to under 800 million. However, the lecture emphasizes nuance: traffic was already declining before ChatGPT due to Google algorithm changes. ChatGPT accelerated a trend rather than creating it.

The gap between “improves productivity” (70%) and “trust the accuracy” (43%) is the central tension of the course. You can be more productive and less trustworthy at the same time if the AI makes you faster at producing code you do not fully understand.

DeepSeek-V3 matched frontier performance for $5.5 million — roughly 1/20th of GPT-4’s cost. The trend shows that algorithmic efficiency (such as Mixture of Experts architectures) can dramatically reduce costs, enabling more players to enter the AI field.

Mistral is a French AI company whose models can be hosted on European infrastructure, complying with GDPR without sending data to US or Chinese servers. This is particularly important for European companies with data sovereignty requirements.

MCP is an open standard that lets AI models connect to external data sources and tools — file systems, databases, APIs, CI/CD systems, and more. The lecture compares it to “what USB was to hardware: a universal connector that replaces proprietary integrations.”

The course chose VS Code + Copilot for five reasons: (1) Free for students, (2) Best IDE integration, (3) Industry standard, (4) Increasingly model-agnostic, and (5) Supports all interaction modes (completions, chat, agent mode). It is NOT the only tool — the course also explores Claude Code, direct API access, and Cursor.