DEV Community: Yuri Penskikh

Django: History and Trends

Yuri Penskikh — Wed, 01 Jul 2026 11:33:44 +0000

Django is a Python development classic that has been helping build and maintain web projects of all scales for two decades. But how does this monolith cope with the challenges of modern async and SPA? And why has strict backward compatibility become both its greatest advantage and its main limitation?

TL;DR: The Django + DRF stack faces serious limits with typing and async. True async in Django is still not fully fixed, but the ecosystem adapts. Developers use new tools like django-modern-rest for better APIs, or just return to SSR + HTMX.

1. The Era of Web Spaghetti: How Django and Its Contemporaries Changed the Game

The mid-2000s marked a global shift in web development. Almost simultaneously, a new generation of frameworks emerged: Ruby on Rails, Symfony, and Django. They brought order to the chaos of the lightweight web segment. They also offered a flexible alternative to the heavy Enterprise stack (Java, .NET).

Spaghetti code—where HTML, SQL queries, and business logic were mixed in a single file—was replaced by a clear separation of concerns and the concept of rapid development.

Django, like its competitors, offered developers everything they needed out of the box:

A built-in ORM
An authentication subsystem
An administrative panel

The web at that time operated on the classic Server-Side Rendering (SSR) model. The server received a request, fetched data from the database, rendered an HTML template, and returned the finished page to the browser. The community embraced pure dynamic typing. Programmers valued Python for its development speed and freedom from compilation.

In this paradigm, the MTV (Model-Template-View) architecture fully met business needs. The built-in admin panel (Django Admin) became a technological breakthrough. Unlike Ruby on Rails, where Scaffolding generated physical code files, Django chose dynamic configuration. The admin panel read model metadata at application startup and generated the management interface on the fly.

However, the decision to bind the admin panel, forms, and ORM via deep dynamic model introspection carried a hidden cost. Years later, this magic would turn into a serious barrier to full static typing support. Furthermore, the monolithic synchronous nature of the old ORM would significantly slow down the framework's transition to a modern asynchronous architecture.

2. The Great Async Irony: How DRF’s Creator Paved the Way for FastAPI

By the mid-2010s, the explosive popularity of AngularJS and React turned Single Page Applications (SPAs) into the new industry standard. The backend became a mere data provider, serving pure JSON. This posed a serious challenge for monolithic Django: its built-in templates and forms suddenly became obsolete.

Tom Christie saved the situation by creating the Django REST Framework (DRF). For many years, this tool secured Django's status as the primary web API in the Python ecosystem. However, DRF's architecture was designed long before modern language standards emerged—leaving it without built-in integration for type hints.

Everything changed with the release of Python 3.5 in 2015. The language officially received native async support via async/await syntax (asyncio) and a type hinting standard. The rise of the mypy static analyzer permanently changed Python development culture. Missing type hints in large projects gradually became bad practice, as codebases without them are difficult to refactor.

Against this background, a major historical irony unfolded. The industry moved toward microservices, where Node.js and Go visibly outperformed Python in highly concurrent I/O operations. While Django Core developer Andrew Godwin tried to catch up with the trend by creating the ASGI async specification for Django, Tom Christie effectively laid a time bomb under his own creation. He founded Encode and released a new async ecosystem completely relying on fresh Python 3.5+ features: the Uvicorn server, the Starlette lightweight framework, and HTTPX.

Starlette soon became the foundation for FastAPI. It combined Tom Christie's async groundwork with strict typing and end-to-end validation via Pydantic. In this light, classic Django, along with the aging DRF, suddenly looked like a bulky tech stack from the past.

3. Technological Deadlock: What Is Wrong with the Old Stack Today?

Caught in the new reality, the Django + DRF combination began rapidly losing ground for modern API design. There are several compelling reasons for this:

Low performance of DRF serializers. Pure Python validation with numerous internal calls and dynamic introspection runs slowly. Modern tools powered by Rust cores (like Pydantic v2) parse and validate data several times faster.
Violation of the Single Responsibility Principle (SRP). The ModelSerializer class is tightly coupled with the ORM. It simultaneously describes the request/response schema, validates business logic, and manages data persistence. In large projects, this often creates bloated "God Objects" spanning thousands of lines of code.
Lack of native async support in DRF. The framework's architecture remains synchronous: serializer and view methods do not support the await keyword. When running a project via an ASGI server, Django's async core is forced to run DRF's synchronous code inside a thread pool, which completely negates the benefits of async.
Difficulties with message broker integration. Django's architecture is isolated within the classic request-response lifecycle. There is no built-in mechanism to run long-lived processes. The django.tasks module introduced in Django 6.0 only defines an interface but lacks a production-ready worker. To work with RabbitMQ or Kafka, developers still have to deploy Celery or write custom management commands.

4. Evolution in Spite of Obstacles: How `django-modern-rest` Changes the Game

The community quickly realized that waiting for a global overhaul of DRF was pointless and began creating alternative solutions. Following the popular Django Ninja, django-modern-rest (DMR) became the technological answer to modern challenges.

The main conceptual difference of DMR is the complete separation of the API layer from business models, thanks to a pluggable architecture. Instead of heavy DRF mechanisms, developers can choose their validation engine: the familiar Pydantic or the ultra-performant msgspec, which accelerates JSON processing significantly.

DMR generates accurate OpenAPI specifications directly from type hints and strictly validates data at runtime. This provides:

Native async support (ASGI) without extra layers
Out-of-the-box auto-documentation
Advanced API testing (e.g., via integration with Schemathesis)

At the same time, standard Django Views remain under the hood. For developers, this means the best of both worlds: FastAPI-level speed and strict typing without losing the mature Django ecosystem.

5. The Burden of Backward Compatibility: Why You Can't Just Rewrite Everything

Why can't Django Core developers rebuild the system to match new standards overnight? The answer is simple: businesses value the framework precisely for its stability. Radical changes would break thousands of third-party packages. By breaking version continuity, Django risks repeating the painful, drawn-out experience the IT industry faced when moving from Python 2 to Python 3.

Consequently, async features are introduced evolutionarily—from release to release. In practice, however, this approach remains a compromise. Under the hood, the framework's async interfaces are still forced to rely on synchronous threads when interacting with legacy components. Additionally, a massive array of third-party plugins, the DRF ecosystem, and the built-in admin panel remain tightly bound to a synchronous architecture.

Conclusion: Will the Old Timer Keep Fighting?

Is Django dying? Definitely not. The framework is going through a prolonged and painful transformation. True, it is no longer the default choice for lightweight microservices. However, the emergence of tools like django-modern-rest clearly proves that the ecosystem can adapt to modern challenges despite the conservative limitations of its core.

Moreover, there is a distinct "SPA fatigue" visible in the industry today. Maintaining a bloated frontend just for simple interfaces costs businesses a lot of money. The industry is returning to Server-Side Rendering (SSR) and lightweight HTMX. In this new reality, Django's classic architecture is once again in high demand.

Ultimately, strict backward compatibility—which once seemed like a limitation—has turned into the framework's main commercial advantage. Projects written ten years ago continue to run stably, generate revenue, and upgrade smoothly to current versions without needing a complete rewrite.

Stop Saying Python Iterators Are Eager

Yuri Penskikh — Wed, 17 Jun 2026 12:40:09 +0000

As a backend developer, I sometimes help companies evaluate candidates by reviewing their recorded technical interviews. However, over time, I’ve noticed a deeply ingrained misconception. When discussing memory management or data streaming, many developers explicitly state:

"Iterators in Python are inherently eager. If you want true lazy loading or lazy evaluation, you have to use generators and the yield keyword."

This misconception is common. Many popular bootcamps and online courses introduce lazy evaluation exclusively through generators. Custom class-based iterators are usually skipped or dismissed as boilerplate-heavy OOP theory rarely used in production Python.

This confusion is further reinforced by two common educational simplifications:

The List vs. Generator Expression Analogy: Beginners are taught that square brackets [...] (list comprehensions) are eager and take up memory, while parentheses (...) (generator expressions) are lazy. This often creates a false binary mental model: "generators = lazy, everything else = eager."
Standard "Textbook" Examples: When courses demonstrate a custom iterator, they usually write a basic class that accepts an already fully loaded list in its __init__ and simply increments an index in __next__. While this is valid for in-memory data, it leads developers to assume that custom iterators inherently require loading all data upfront.

In reality, generators are a specialized language feature designed to implement the iterator protocol automatically. They comply with the exact same interface (__iter__ and __next__). A generator is lazy not because of some magical property of the yield keyword, but simply because it adheres to this underlying contract.

To show that custom iterators can be lazy without using any generators or yield keywords, I’ve put together a lightweight and reproducible benchmark.

🧪 The Experiment: Proving Lazy Loading with Custom Iterators

Suppose we need to read a database export file (test_users_db.txt) containing 1_000_000 rows of user data. We want to iterate over these users but stop immediately once we find a specific record (at HALF_ROWS=500_000). If custom iterators were truly "eager," our lazy implementation would significantly increase RAM consumption.

💡 Please note: I’m using a local text file here as a simplified data source. However, in a production system, this exact same pattern is used to stream rows from a heavy SQL database cursor, fetch paginated data from an external HTTP API, or consume messages from a queue.

Let's look at the code:

from __future__ import annotations
import tracemalloc
from dataclasses import dataclass
from pathlib import Path

FILE_PATH = Path("test_users_db.txt")
TOTAL_ROWS = 1_000_000
HALF_ROWS = TOTAL_ROWS // 2


def generate_test_file(path: Path, rows: int = TOTAL_ROWS) -> None:
    if not path.exists():
        with open(path, "w", encoding="utf-8") as f:
            for i in range(1, rows + 1):
                f.write(f"{i},User_Name_{i}\n")


@dataclass(frozen=True)
class UserDTO:
    id: int
    name: str


# --- LAZY IMPLEMENTATION ---
class LazyUserGateway:
    def __init__(self, file_object):
        self.file = file_object

    def __iter__(self) -> LazyUserIterator:
        return LazyUserIterator(self.file)


class LazyUserIterator:
    def __init__(self, file_object):
        self.file = file_object

    def __iter__(self) -> LazyUserIterator:
        return self

    def __next__(self) -> UserDTO:
        # Step-by-step stream: reading only one line into memory at a time
        line = self.file.readline()
        if not line:
            raise StopIteration

        user_id, name = line.strip().split(",")
        return UserDTO(id=int(user_id), name=name)


# --- EAGER IMPLEMENTATION ---
class EagerUserGateway:
    def __init__(self, file_object):
        self.file = file_object

    def __iter__(self) -> EagerUserIterator:
        return EagerUserIterator(self.file)


class EagerUserIterator:
    def __init__(self, file_object):
        # The readlines() call forces Python to build a massive list of strings in RAM upfront
        self.lines = file_object.readlines()
        self.index = 0

    def __iter__(self) -> EagerUserIterator:
        return self

    def __next__(self) -> UserDTO:
        if self.index >= len(self.lines):
            raise StopIteration

        line = self.lines[self.index]
        self.index += 1

        user_id, name = line.strip().split(",")
        return UserDTO(id=int(user_id), name=name)


# --- BENCHMARKS ---
def test_eager_gateway() -> float:
    tracemalloc.start()

    with open(FILE_PATH, "r", encoding="utf-8") as f:
        gateway = EagerUserGateway(f)
        for user in gateway:
            if user.id == HALF_ROWS:
                print(f"[Eager] Final reached: {user}")
                break

    current, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return peak / (1024 * 1024)


def test_lazy_gateway() -> float:
    tracemalloc.start()

    with open(FILE_PATH, "r", encoding="utf-8") as f:
        gateway = LazyUserGateway(f)
        for user in gateway:
            if user.id == HALF_ROWS:
                print(f"[Lazy]  Final reached: {user}")
                break

    current, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return peak / (1024 * 1024)


if __name__ == "__main__":
    generate_test_file(FILE_PATH)
    print("\n--- Memory Benchmark Running ---")

    peak_eager = test_eager_gateway()
    print(f"-> Peak Memory (EagerUserGateway): {peak_eager:.2f} MB\n")

    peak_lazy = test_lazy_gateway()
    print(f"-> Peak Memory (LazyUserGateway):  {peak_lazy:.2f} MB\n")

    if FILE_PATH.exists():
        FILE_PATH.unlink()

📊 Benchmark Results

Running the script yields the following terminal output:

--- Memory Benchmark Running ---
[Eager] Final reached: UserDTO(id=500000, name='User_Name_500000')
-> Peak Memory (EagerUserGateway): 69.97 MB

[Lazy]  Final reached: UserDTO(id=500000, name='User_Name_500000')
-> Peak Memory (LazyUserGateway):  0.15 MB

This represents a ~500x reduction in RAM consumption (dropping from ~69.97MB to 0.15MB). While the eager implementation scales linearly with the dataset size, the lazy iterator maintains a flat, near-zero memory footprint regardless of the row count.

🧠 Iterators vs. Generators: Contract vs. Convenience

In practice, generators are the industry standard for a reason. In most daily tasks, writing a quick generator function with yield is much faster, safer, and cleaner than writing custom iterator classes. The generator object itself handles the boilerplate, implicitly saves execution state, and natively facilitates resource cleanup.

But as engineers, we shouldn't confuse implementation-specific capabilities with a structural interface contract, especially when designing architectural boundaries:

An Iterator is just a contract (interface). It guarantees that data can be traversed step-by-step via __next__. How that data is managed — whether the iterator eagerly fetches it upfront, lazily streams it from an API, or simply acts as a pointer to data passed to it from the outside — is entirely up to the developer who implements it. An iterator doesn't even need to own or fetch data; it just drives the traversal logic.
A Generator is a specialized language feature that auto-implements this protocol while supporting additional lifecycle and communication methods (via .send(), .throw(), and .close()). For basic data streaming, you don't need this additional complexity. A custom iterator class gives you a clean, one-way contract strictly limited to data traversal, keeping your architectural boundaries well-defined.

Custom class-based iterators deliver the exact same polymorphic interface. They aren't meant to replace generators everywhere. Instead, they prove that lazy evaluation is an architectural design choice, not a magic trick powered strictly by yield.

🏁 Conclusion

The "iterators are eager" take is just a byproduct of simplified educational content. Diving deeper into core protocols helps us write much better, more predictable architectural layers.

💬 What about you? Have you ever heard that custom Python iterators are inherently eager?