Read OSS

From URL to Execution: Deno's Module Loading, Resolution, and TypeScript Pipeline

Advanced

Prerequisites

  • Articles 1-2
  • JavaScript module systems (ESM, CJS, import maps)
  • Basic understanding of TypeScript compilation

From URL to Execution: Deno's Module Loading, Resolution, and TypeScript Pipeline

In Articles 1 and 2, we saw how the CLI dispatches subcommands and how deno_core bridges Rust to JavaScript through extensions and ops. But when you run deno run https://example.com/mod.ts, something remarkable happens: Deno fetches a TypeScript file from the internet, resolves its imports (which might be URLs, npm packages, or JSR modules), transpiles everything, and executes it — all without a package.json or build step. This article dissects that pipeline: the zoo of specifier types, the layered resolver stack, the module graph pre-analysis, and the dual TypeScript type-checking system that's currently transitioning from a JavaScript-based tsc to a Go-based tsgo.

The Module Specifier Zoo

Deno handles a wider variety of module specifiers than any other JavaScript runtime. Where Node.js deals primarily with bare specifiers and relative paths, Deno must resolve:

Specifier Type Example Resolved By
File URL file:///home/user/mod.ts Direct filesystem access
HTTPS URL https://deno.land/std/path/mod.ts HTTP fetch + cache
Import map entry std/path → mapped URL Import map resolution
JSR specifier jsr:@std/path@1.0.0 JSR registry resolution
npm specifier npm:express@4 npm registry + node_modules
Bare specifier lodash Package.json, import map, or error
Data URL data:text/javascript,... Inline parsing
Node built-in node:fs ext/node polyfills
graph TD
    SPEC[Module Specifier] --> TYPE{Specifier Type?}
    TYPE -->|file://| FS[Read from filesystem]
    TYPE -->|https://| HTTP[Fetch + HTTP cache]
    TYPE -->|jsr:| JSR[JSR registry lookup]
    TYPE -->|npm:| NPM[npm resolution]
    TYPE -->|node:| NODE[ext/node polyfill]
    TYPE -->|import map| IMAP[Import map transform]
    TYPE -->|bare| BARE{Has package.json<br/>or import map?}
    BARE -->|Yes| IMAP
    BARE -->|No| ERR[Error: Module not found]
    IMAP --> TYPE
    JSR --> HTTP

The libs/resolver/lib.rs crate is the extracted resolver library. It imports from node_resolver for Node.js-compatible resolution and from deno_package_json for package.json parsing, then layers its own resolution strategies on top.

The Resolver Stack

Module resolution in Deno isn't a single function — it's a stack of resolvers, each handling a different concern. The CLI wires these together through type aliases in cli/resolver.rs:

pub type CliResolver = deno_resolver::graph::DenoResolver<
  DenoInNpmPackageChecker,
  DenoIsBuiltInNodeModuleChecker,
  CliNpmResolver,
  CliSys,
>;

The DenoResolver is generic over four type parameters, allowing different implementations for CLI, standalone binaries, and testing. The resolution pipeline operates in layers:

flowchart TD
    INPUT["import 'foo/bar'"]
    IM["1. Import Map Resolution<br/>Maps bare specifiers to URLs"]
    JSR["2. JSR Resolution<br/>jsr: → registry URL"]
    NPM["3. npm Resolution<br/>npm: → node_modules path"]
    CJS["4. CJS/ESM Detection<br/>package.json type field"]
    SLOPPY["5. Sloppy Imports<br/>Try .ts, .js, /index.ts"]
    NODE_RES["6. Node Resolution<br/>node_modules traversal"]
    FINAL["Resolved URL"]

    INPUT --> IM
    IM --> JSR
    JSR --> NPM
    NPM --> CJS
    CJS --> SLOPPY
    SLOPPY --> NODE_RES
    NODE_RES --> FINAL

"Sloppy imports" is Deno's concession to developer ergonomics — when enabled, it tries appending .ts, .js, /index.ts, etc., mimicking the resolution behavior developers expect from bundlers. The resolver emits warnings when sloppy resolution kicks in, encouraging explicit imports.

Tip: The CliCjsTracker type tracks whether a module should be treated as CJS or ESM. This is crucial for npm compatibility: many npm packages use require() internally, and Deno needs to detect and handle this at module load time rather than execution time.

CliModuleLoaderFactory and the ModuleLoader Trait

The cli/module_loader.rs file implements deno_core::ModuleLoader — the trait that V8 calls when it needs to resolve and load a module. The file is ~1,500 lines because module loading touches nearly every subsystem: file fetching, transpilation, npm resolution, code caching, and graph analysis.

The load flow for a single module looks like this:

sequenceDiagram
    participant V8 as V8 Engine
    participant ML as ModuleLoader
    participant Graph as Module Graph
    participant Fetch as File Fetcher
    participant Trans as Transpiler
    participant Cache as Code Cache

    V8->>ML: resolve(specifier, referrer)
    ML->>Graph: Check prepared graph
    Graph-->>ML: Resolved specifier
    V8->>ML: load(specifier)
    ML->>Graph: Lookup in graph
    alt In graph (pre-loaded)
        Graph-->>ML: Source + media type
    else Not in graph
        ML->>Fetch: Fetch module
        Fetch-->>ML: Raw source
    end
    ML->>Trans: Transpile if TypeScript
    Trans-->>ML: JavaScript source
    ML->>Cache: Check V8 code cache
    Cache-->>ML: Cached bytecode (if any)
    ML-->>V8: ModuleSource { code, code_cache }

The ModuleLoader implementation differentiates between modules that were pre-analyzed in the module graph (the happy path for deno run) and modules loaded dynamically via import(). Pre-analyzed modules are already fetched and transpiled; dynamic imports may trigger new network requests.

The code cache integration is particularly interesting. Deno stores V8's compiled bytecode alongside module source code, keyed by a hash of the source content. On subsequent runs, the cached bytecode is loaded, and V8 can skip the parse-and-compile phase entirely. This is separate from the HTTP cache — it's a cache of V8's internal representation.

The Module Graph: Pre-Analyzing Dependencies

Before executing any code, Deno builds a complete dependency graph using deno_graph. The cli/graph_util.rs module orchestrates this:

flowchart LR
    ROOT["Root module<br/>hello.ts"] --> BUILD["ModuleGraphBuilder<br/>.build()"]
    BUILD --> FETCH["Parallel fetch<br/>all dependencies"]
    FETCH --> PARSE["Parse imports<br/>(deno_ast)"]
    PARSE --> RESOLVE["Resolve specifiers<br/>(deno_resolver)"]
    RESOLVE --> RECURSE{More deps?}
    RECURSE -->|Yes| FETCH
    RECURSE -->|No| GRAPH["Complete<br/>ModuleGraph"]
    GRAPH --> CHECK["Type check<br/>(optional)"]
    GRAPH --> PREP["ModuleLoadPreparer<br/>makes graph available<br/>to ModuleLoader"]
    CHECK --> EXEC["Execute"]
    PREP --> EXEC

The ModuleGraph from the deno_graph crate is built by walking imports recursively. Each module is fetched (potentially over HTTP), parsed to extract its import/export statements, and its dependencies are resolved and queued. This happens in parallel — multiple HTTP requests fly concurrently, and CPU-bound parsing is interleaved.

The graph serves multiple purposes:

  • Parallel fetching: All modules are fetched before execution starts, avoiding waterfall delays
  • Type checking: The type checker needs the full graph to resolve cross-module type references
  • Error reporting: Missing modules, circular dependencies, and resolution failures are reported before any code runs
  • Lock file validation: Module integrity hashes are checked against the lock file

The ModuleLoadPreparer bridges the graph and the ModuleLoader — it ensures the graph is built, optionally type-checked, and then stored where the ModuleLoader can access it during V8's module evaluation.

TypeScript Compilation: tsc vs tsgo

Deno's type checking system is in the middle of an architectural transition. The cli/type_checker.rs module orchestrates both backends.

The legacy system (cli/tsc/mod.rs) runs the JavaScript-based TypeScript compiler inside a dedicated V8 isolate. Deno provides a custom CompilerHost implementation via ops that the TypeScript compiler calls to resolve modules, read files, and write output. It's essentially a TypeScript compiler embedded within a JavaScript runtime embedded within a Rust program.

The new system (cli/tsc/go.rs) uses tsgo — a Go-based TypeScript type checker that communicates with Deno via RPC. The comment at line 45 reveals a pragmatic workaround:

// the way tsgo currently works, it really wants an actual tsconfig.json file.
// it also doesn't let you just pass in root file names. instead of making more
// changes in tsgo, work around both by making a fake tsconfig.json file with
// the "files" field set to the root file names.

The Go client creates a synthetic tsconfig.json in memory, passes it to tsgo as if it were a real file, and handles module resolution callbacks from Go back into Rust via an RPC channel (SyncRpcChannel).

sequenceDiagram
    participant TC as TypeChecker
    participant Legacy as tsc (JS in V8)
    participant Go as tsgo (Go via RPC)
    
    TC->>TC: Check DENO_USE_TSGO env var
    alt Legacy mode
        TC->>Legacy: Create V8 isolate
        Legacy->>Legacy: Load TypeScript compiler
        Legacy->>TC: Request modules (via ops)
        TC-->>Legacy: Module sources
        Legacy-->>TC: Diagnostics
    else tsgo mode
        TC->>Go: Spawn tsgo process
        Go->>TC: ResolveModuleName callback
        TC-->>Go: Resolved specifier
        Go->>TC: ReadFile callback
        TC-->>Go: File contents
        Go-->>TC: Diagnostics
    end

It's important to note that type checking is completely separate from transpilation. TypeScript is transpiled (stripped of types) by deno_ast using swc during module loading — this is fast and always happens. Type checking is optional (--no-check skips it) and expensive — it's the part that validates types are correct across the entire module graph.

Tip: The TypeCheckMode enum has three variants: None (skip checking), Local (check local files only, skip node_modules), and All (check everything including dependencies). Local is the default for deno check — use --all to check dependencies too.

File Fetching and Caching

The cli/file_fetcher.rs module handles fetching modules from any source — local files, HTTP URLs, or data URLs. For remote modules, it integrates with deno_cache_dir for HTTP caching:

flowchart TD
    REQ["Fetch request<br/>https://deno.land/std/path/mod.ts"]
    LOCAL{Local file?}
    REQ --> LOCAL
    LOCAL -->|Yes| READ[Read from disk]
    LOCAL -->|No| CACHE{In HTTP cache?}
    CACHE -->|Yes, fresh| HIT[Return cached]
    CACHE -->|Yes, stale| REVALIDATE[Conditional GET<br/>If-None-Match / If-Modified-Since]
    CACHE -->|No| FETCH[HTTP GET]
    REVALIDATE -->|304| HIT
    REVALIDATE -->|200| STORE[Store in cache]
    FETCH --> STORE
    STORE --> DECODE[Detect charset<br/>Decode to UTF-8]
    READ --> DECODE
    HIT --> DECODE
    DECODE --> FILE["TextDecodedFile<br/>{specifier, source, media_type}"]

The TextDecodedFile struct is the unified output: a specifier, decoded source text, and a MediaType (TypeScript, JavaScript, JSX, JSON, etc.) determined from HTTP headers or file extension. The media type drives subsequent transpilation — TypeScript files go through swc, JavaScript files pass through unchanged.

The CacheSetting enum controls caching behavior: Use (default — use cache, fetch if missing), Only (offline mode — error if not cached), ReloadAll (ignore cache, re-fetch everything), and ReloadSome (selective reload). The --cached-only and --reload flags map to these settings.

What's Next

We've traced a module's journey from specifier string to executed JavaScript: resolution through a layered stack, parallel fetching into a dependency graph, optional type checking through either a JS-based or Go-based TypeScript compiler, transpilation via swc, and execution in V8 with code caching. In the next article, we'll zoom into the MainWorker itself — how it's created, how the bootstrap sequence constructs the Deno global namespace from extension modules, and how the 8-type permissions system enforces security at the op boundary.