Choosing the API Runtime: Matching the Language to the Workload

Over the span of my API development career, I have seen shifts in programming language of choice, the emergence of API toolkits and frameworks, and true evolution in the raw compute power of frontend devices that shape the purpose of an API. My early work was in PHP, across Laravel, Lumen, and Symfony. From there I moved into Python, where my experience runs deepest. Its web story evolved across several generations, beginning with Apache's mod_python and then the WSGI and uWSGI execution paradigms; layered over those came a succession of API frameworks, a good many of which I worked in (Django, web2py, Pylons, Tornado, and Flask, roughly in that order), and presently FastAPI. More recently my work has taken me into Rust, first with Actix and now with Axum, on the secure, high-performance core of an open source CMS we are presently building. These experiences shape how I have come to think about a question I am asked often: which language, or which runtime, should a new API be built on?

A quick aside might be in order. In the early days the server did nearly all of the work and shipped a finished page to a comparatively thin client, whose job was mostly to display what arrived. As client devices grew more capable (first browsers with real scripting engines, then smartphones carrying real compute of their own), a good deal of the rendering, state, and interaction moved onto the device in front of the user, and the purpose of an API shifted with it. Where an API once returned an assembled page, it now tends to return data over a clean contract and leave presentation to a capable client. That change is as much a reason the landscape looks the way it does today as any shift of language or framework. It directly impacts the runtime question, since a data-serving API is judged far more on how well it handles concurrency and I/O than on how quickly it can assemble a page.

We so often lead with the 'which language to adopt' question when embarking on a new project, and that instinct can do us a disservice. Arguably, a more valuable initial question may not be "Rust or Python?" but rather "what is this service doing, and who is building it?" Programming language selection should reflect the security and performance needs of the workload, the skillset of the team, and the project velocity requirements. Comparing on a single axis (raw throughput, most often) manufactures a winner and a loser out of two languages that lead on different ground. There is an old sales adage: "When the only issue is price, there will be a winner and a loser." By stepping back and viewing the entire requirements landscape, a much richer picture comes into focus. For most APIs, which spend the vast majority of their time waiting on a database, a high-velocity runtime like Python/FastAPI is the sound choice. A compiled runtime makes complete sense when the work is CPU-bound at scale, when tail-latency and memory footprint are of paramount concern, or when the core is security-critical enough that compile-time guarantees pay for themselves. An extremely compelling, though frequently unexplored option is to consider whether a project might warrant two languages woven together, each used for the tasks where it is a truer fit.

A Richer Set of Factors than Speed Alone

By focusing strictly on performance, as is often done, we can find ourselves unwittingly moving toward a decision that might hurt our project in other areas. In reality it is one of several factors. For the vast majority of projects I have been involved in, I have found that the other factors were often ignored at the outset, resulting in a codebase that was simply unable to deliver on the original intent.

Delivery speed and ease of implementation come first. Python is built for velocity: dynamic typing, a shallow ramp, fast iteration, and in FastAPI combined with Pydantic a contract-first framework with almost no boilerplate. Rust trades that for compile-time guarantees: a steeper learning curve, slower compiles, more ceremony before the first feature ships. Across a broad, fast-moving surface, Python delivers in a fraction of the time.

The talent pool and team scale set the practical limit. Python is among the most widely known languages on earth; commonly cited figures put open Python roles at roughly ten to twenty times the number of Rust roles, and Python developers are both plentiful and quick to onboard. Rust's pool is small, its hires are expensive, and searches run long. A large project that must move quickly with a large team is effectively off the table in Rust: it cannot be staffed fast enough, and the ramp would tax even the developers who do come on board. Rust fits a small, senior team on a bounded, critical surface, not a fifty-person delivery organization racing a roadmap.

The ecosystem and available third-party libraries matter as well. Python's library catalog is vast and proven across web, data, scientific computing, and machine learning. Rust's ecosystem (tens of thousands of crates on crates.io, async-first at its core, and growing fast) is strong and improving, but younger and narrower. For many domains a mature, well-documented Python library already exists where the Rust equivalent is nascent or absent, and integrating with the systems already in place is smoother.

Finally, there is the question of where the domain's knowledge lives. Some fields have a center of gravity, and AI/ML is Python's. The research, the teaching, the notebooks (Jupyter/ipynb used for both design and implementation), and the frameworks (PyTorch, scikit-learn, the Hugging Face stack) are all Python-first. Building there in another language means working against the entire body of knowledge. The examples, the tutorials, the available hires, the answered questions all assume Python, and contending with that costs more than it returns.

None of this makes one language the better choice in a general sense; each is well suited to its own ground, and those grounds do not overlap in the way a raw throughput comparison would imply.

Framing the Performance Gap Realistically

When performance is the deciding factor, the single most useful distinction is CPU-bound versus I/O-bound work, because it determines whether the runtime is even on the critical path.

CPU-bound work (tokenization, encoding, compression, cryptography, large-scale data processing, similarity math) is where a compiled language runs an order of magnitude or more faster than interpreted Python; commonly cited figures range from roughly 10x to 100x. The reasons are structural, not tuning gaps: CPython's interpretation overhead, dynamic typing, and the Global Interpreter Lock (one core for CPU-bound threads) are properties of the language, and the experimental free-threaded mode in recent Python versions narrows but does not close the gap with native code.

I/O-bound work is the more common case: the typical CRUD API that reads from Postgres, shapes a little data, and returns JSON. Both runtimes spend nearly all their time waiting on the database, so the language overhead is largely irrelevant. For a database-bound request the performance gap commonly collapses to roughly 1.2x. A Python API that spends 95% of its time on I/O is not meaningfully constrained by Python's speed.

The practical implication: the performance argument for a compiled runtime is contingent entirely on whether the code does real CPU work. Focusing strictly on throughput numbers can lead a team to consider a complete rewrite where perhaps a more selective, surgical modification would be more in keeping with the project's full set of requirements.

The Ground Python and FastAPI Cover Well

Most APIs are database-bound CRUD, and that is the ground Python covers well. The bottleneck is the query plan and the network round-trip, not the interpreter, and FastAPI with Pydantic gives contract-first validation on every endpoint with almost no boilerplate. Below a few thousand requests per second with no strict per-request CPU cost, Python is almost certainly fast enough, and its operational simplicity counts for a great deal.

Teams that are large and facing an aggressive roadmap will also find Python the right choice. Developers are more readily available and can be onboarded quickly; a large Rust delivery organization is impractical to staff and slow to ramp, so what matters is velocity across a broad surface, and Python supplies it.

AI/ML work is a third case where Python is the clear fit. The heavy compute already runs in native code (CUDA/torch, native libraries), Python's overhead around it is modest, and the entire body of knowledge (frameworks, notebooks, and available hires) is Python-first. Iteration speed is what matters most here, and that is squarely Python's strength.

Where a Compiled Runtime Justifies the Cost

The case for a compiled runtime (Rust/Axum, or Go) rests on three distinct conditions, and being precise about them matters, because the decision is easy to make prematurely.

The first is CPU-bound work at scale: tokenization on every request, inference serving, data-pipeline transforms over large volumes, compression or cryptography in the hot path. This is where the order-of-magnitude performance difference is actually felt, and where a compiled runtime repays its added cost.

The second is when tail latency and memory footprint are SLAs rather than nice-to-haves. No garbage-collection pauses means predictable p99 latency; a small static binary means high container density and viable serverless or edge deployment (millisecond cold starts versus seconds for a Python process with its dependency tree).

The third is when the core is security-critical enough that compile-time guarantees justify the additional cost. When "did we remember to scope this?" can be made non-compilable rather than merely tested, that is a different class of assurance entirely (see the companion piece, Rust for the Secure Core).

Go deserves a fair mention here: it offers an excellent operations story and native concurrency with a far gentler ramp than Rust, a deeper talent pool, and fast compiles. It is often the right compiled choice when the decisive need is throughput and simple deployment rather than the strongest possible safety invariants. Go has its own Python-bridge story as well: gopy generates a CPython extension from a Go package, and cgo's c-shared build mode exports a library Python can call through ctypes. The bridge is less smooth than PyO3, though. The cgo boundary carries per-call overhead and a global lock, so it favors batched calls over the fine-grained hot-path weaving that PyO3 makes near-free.

Weaving Python and Rust into a Unified Whole

As mentioned earlier, sometimes leveraging the strengths of two languages together will yield the best result. In the case of blending Python and Rust, a toolkit like PyO3 can be of immense benefit, moving the runtime decision from a single system-level choice down to the individual hot path.

PyO3 is an open-source Rust crate that bridges Rust and the CPython interpreter in both directions. It lets you write a native Python extension module in Rust, compiled to a shared library (.so / .pyd) that Python imports with a plain import statement. It can also embed a Python interpreter inside a Rust binary to run scripts and exchange data. For anyone coming to it for the first time, three capabilities stand out: automatic type conversion (data types translate across the boundary on their own, so a Rust i64 arrives in Python as an int, and so on); safe GIL management (PyO3 exposes abstractions that handle Python's Global Interpreter Lock correctly, so concurrency across the two languages stays sound); and procedural macros (attributes like #[pymodule], #[pyclass], and #[pyfunction] expose ordinary Rust as idiomatic Python with little boilerplate, so the Rust function looks like a normal Python function from the caller's side).

A small companion ecosystem makes it production-ready. Maturin is a near-zero-configuration build tool that compiles a Rust crate straight into a distributable Python wheel, and rust-numpy binds Rust to NumPy's array API for fast multidimensional math. Developers seldom use PyO3 entirely on its own; these tools handle the build-and-distribute path.

The productive approach is to profile first. Roughly 5% of the code typically accounts for the vast majority of CPU cost; that hot path is the candidate for a Rust extension behind a PyO3 boundary, while Python's ecosystem and developer experience remain everywhere else. In fact, this is the architecture behind the most performant Python tools in production today. Polars, Pydantic v2, Hugging Face Tokenizers, and Ruff all keep a Python API over a Rust engine. A complete rewrite is rarely what the situation calls for; isolating that hot path behind a targeted extension is usually all it takes.

In Summary

The aim is to match the runtime to the workload and the team, and to keep the boundary between components clean so the choice stays reversible as either one changes. A live example of the approach: a system can opt for a hardened compiled core on the trust boundary and the latency-critical path while an approachable Python layer handles AI/ML features and plugin experimentation outside it, each language placed where its strengths land, neither declared the universal winner. Reducing the question to raw speed manufactures a winner and a loser out of that picture; a wider accounting across performance, delivery speed, hiring, ecosystem, and the domain's center of gravity lets the right answer fall out of what the service does and who is building it, rather than what happens to be in fashion.

I will leave it there for now, though there is more to each branch of this than a single piece can hold. I can tell you, for instance, of a core where we made cross-tenant access a compile-time impossibility in Rust, and of other systems where a few lines of FastAPI were all the situation called for, but those are conversations for another day.