Unpacking Node.js's Built-in Fetch and its Undici Foundation

Introduction

For years, making HTTP requests in Node.js often involved external libraries like axios or node-fetch. While these libraries have served the community well, they introduced an extra dependency and sometimes a slight learning curve for developers accustomed to the browser's native fetch API. With the release of Node.js 18, a significant milestone was reached: a global, built-in fetch API, mirroring its browser counterpart, became available without needing any external installations. This integration dramatically streamlines web development in Node.js, offering a familiar interface for data fetching and promising improved performance and stability. This article will thoroughly explore Node.js's native fetch, uncovering its underlying architecture, and specifically examining its profound relationship with undici, the cutting-edge HTTP/1.1 client that powers it.

Core Concepts Explained

Before we dive into the specifics of fetch and undici, let's clarify some fundamental terms that will be central to our discussion:

• fetch API: A modern, promise-based API for making network requests, often used to retrieve resources across the network. It's designed to be more flexible and powerful than XMLHttpRequest.
• undici: A high-performance, WHATWG fetch API-compatible HTTP/1.1 client for Node.js, built from the ground up to be fast and reliable. It aims to be the standard HTTP client for Node.js.
• WHATWG fetch Standard: A specification by the Web Hypertext Application Technology Working Group (WHATWG) that defines the fetch API. Node.js's built-in fetch aims to align closely with this standard.
• Streams: A fundamental concept in Node.js for handling data in chunks. fetch leverages streams for both request bodies and response bodies, enabling efficient processing of large amounts of data.
• Request/Response Objects: The core objects used by the fetch API. A Request object represents an outgoing network request, and a Response object represents an incoming network response.

The Inner Workings of Node.js Fetch

Node.js's built-in fetch API is not a completely independent implementation. Instead, it is a direct re-export and slight modification of the undici library. This strategic choice by the Node.js core team offers several advantages:

1. Standard Compliance: undici is meticulously designed to comply with the WHATWG fetch standard, ensuring that Node.js developers get an API that behaves consistently with browser environments. This reduces cognitive overhead and facilitates isomorphic code.
2. Performance and Efficiency: undici is renowned for its exceptional performance. It features a custom request/response parser, connections pooling, pipelining, and other optimizations that significantly reduce overhead and improve throughput compared to older HTTP clients. By leveraging undici, Node.js fetch inherits these performance benefits.
3. Active Development: undici is an actively developed project, ensuring continuous improvements, bug fixes, and feature enhancements. Integrating it allows Node.js to quickly adopt these advancements.

Basic Usage Example

Using fetch in Node.js 18+ is as straightforward as in the browser:

// my-data-fetcher.js

async function fetchData(url) {
  try {
    const response = await fetch(url);

    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }

    const data = await response.json(); // or .text(), .blob(), .arrayBuffer(), .formData()
    console.log('Fetched data:', data);
    return data;
  } catch (error) {
    console.error('Error fetching data:', error);
    throw error;
  }
}

// Example usage:
fetchData('https://jsonplaceholder.typicode.com/todos/1')
  .then(data => console.log('Successfully retrieved:', data))
  .catch(error => console.error('Failed to retrieve:', error.message));

// Example with POST request and custom headers
async function postData(url, data) {
  try {
    const response = await fetch(url, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Accept': 'application/json'
      },
      body: JSON.stringify(data)
    });

    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }

    const result = await response.json();
    console.log('POST successful:', result);
    return result;
  } catch (error) {
    console.error('Error posting data:', error);
    throw error;
  }
}

postData('https://jsonplaceholder.typicode.com/posts', {
    title: 'foo',
    body: 'bar',
    userId: 1,
  })
  .then(data => console.log('New post created:', data))
  .catch(error => console.error('Failed to create post:', error.message));

To run this code, simply save it as my-data-fetcher.js and execute node my-data-fetcher.js in your terminal. You'll observe the data being fetched and logged to the console.

How undici Underpins fetch

When you call global.fetch() in Node.js 18+, what you're essentially invoking is undici.fetch(). The integration is seamless. undici provides the core HTTP client functionality, handling connection management, request serialization, response parsing, and error handling.

Let's look at a simplified conceptual flow:

1. fetch Call: When fetch(url, options) is called, the global fetch function (which is undici.fetch) receives these arguments.
2. Request Object Creation: undici internally constructs a Request object based on the provided URL and options, adhering to the WHATWG fetch specification.
3. Connection Management: undici utilizes a sophisticated connection pooling mechanism. It reuses existing HTTP/1.1 connections or establishes new ones efficiently.
4. Request Sending: The Request object is serialized into an HTTP message and sent over the established connection. undici handles concerns like headers, body encoding, and redirects.
5. Response Receiving and Parsing: Upon receiving the server's response, undici rapidly parses the incoming HTTP message, constructs a Response object, and makes the response body available as a ReadableStream.
6. Response Object Return: The fetch call resolves with the Response object, allowing you to access properties like status, headers, and methods like json() or text() to consume the body. These body methods often involve consuming the underlying undici stream.

Advanced fetch features and undici's role

undici exposes a more low-level API beyond just fetch, which can be accessed if direct control over HTTP client behavior is needed. For example, undici.Agent offers fine-grained control over connection pooling, timeouts, and redirects.

While global.fetch often suffices, for scenarios requiring custom connection agents or advanced pooling strategies, direct undici usage might be considered. However, the fetch API itself provides a robust set of options within its init object that cover most common use cases, such as:

• signal: Aborting requests using AbortController.

  const controller = new AbortController();
  const timeoutId = setTimeout(() => controller.abort(), 5000); // Abort after 5 seconds
  
  fetch('https://slow-api.example.com/data', { signal: controller.signal })
    .then(response => {
      clearTimeout(timeoutId);
      return response.json();
    })
    .then(data => console.log(data))
    .catch(err => {
      if (err.name === 'AbortError') {
        console.error('Fetch aborted by user or timeout');
      } else {
        console.error('Fetch error:', err);
      }
    });

• redirect: Controlling redirect behavior (follow, error, manual).
• body with Streams: Sending large amounts of data efficiently without buffering the entire payload in memory.

  const { Readable } = require('stream');
  
  async function uploadStreamData() {
      const readableStream = new Readable({
          read() {
              this.push('Hello ');
              this.push('World!');
              this.push(null); // No more data
          }
      });
  
      try {
          const response = await fetch('https://httpbin.org/post', {
              method: 'POST',
              headers: { 'Content-Type': 'text/plain' },
              body: readableStream // Node.js fetch can directly accept Readable streams
          });
  
          const data = await response.json();
          console.log('Stream upload response:', data);
      } catch (error) {
          console.error('Stream upload error:', error);
      }
  }
  
  uploadStreamData();

Node.js's fetch API, thanks to undici, supports these advanced mechanisms, making it a powerful and versatile tool for network communication in server-side applications.

Conclusion

The integration of the fetch API into Node.js 18+ marks a significant step forward for the platform, offering developers a familiar, powerful, and performant way to make HTTP requests. This seamless experience is largely attributable to undici, the cutting-edge HTTP/1.1 client that acts as the backbone of Node.js's native fetch. By leveraging undici, Node.js fetch not only achieves high performance and strict adherence to the WHATWG fetch standard but also provides a robust foundation for modern web development. Node.js fetch simplifies HTTP interactions, allowing developers to focus more on application logic and less on client-side HTTP complexities.