Defer and Stream

Stream and defer are directives that allow you to improve latency for clients by sending data the most important data as soon as it's ready.

As applications grow, the GraphQL operation documents can get bigger.. The server will only send the response back once all the data requested in the query is ready. However, not all requested data is of equal importance, and the client may not need all of the data at once. To remedy this, GraphQL specification working group is working on introducing new @defer and @stream directives which allows applications to request a subset of data which is critical and get the rest of the data in subsequent responses from the server. This proposal is in Stage 2, meaning GraphQL libraries can start implementing this as experimental feature to provide feedback to the working group.

💡

Stream and Defer are experimental features and not yet stable. The implementation can and will change. Furthermore, there is no yet a stable specification for the incremental delivery protocol.

Installation

Enabling support for the @defer and @stream directive requires installing a plugin.

yarn add @graphql-yoga/plugin-defer-stream

Quick Start

The following schema shows a schema whose field resolvers are slow

Slow Schema

import { createYoga, createSchema } from 'graphql-yoga'
import { createServer } from 'node:http'
import { useDeferStream } from '@graphql-yoga/plugin-defer-stream'
 
const typeDefs = /* GraphQL */ `
  type Query {
    alphabet: [String!]!
    """
    A field that resolves fast.
    """
    fastField: String!
 
    """
    A field that resolves slowly.
    Maybe you want to @defer this field ;)
    """
    slowField(waitFor: Int! = 5000): String
  }
`
 
const wait = (time: number) =>
  new Promise((resolve) => setTimeout(resolve, time))
 
const resolvers = {
  Query: {
    async *alphabet() {
      for (const character of ['a', 'b', 'c', 'd', 'e', 'f', 'g']) {
        yield character
        await wait(1000)
      }
    },
    fastField: async () => {
      await wait(100)
      return 'I am speed'
    },
    slowField: async (_, { waitFor }) => {
      await wait(waitFor)
      return 'I am slow'
    }
  }
}
 
const yoga = createYoga({
  schema: createSchema({
    typeDefs,
    resolvers
  }),
  plugins: [useDeferStream()]
})
 
const server = createServer(yoga)
 
server.listen(4000, () => {
  console.info('Server is running on http://localhost:4000/graphql')
})

Create this file and start the GraphQL server.

Using Defer

The @defer directive allows you to post-pone the delivery of one or more (slow) fields grouped in an inlined or spread fragment.

Visit http://localhost:4000/graphql and paste the following operation into the left panel.

GraphQL Operation using @defer

query SlowAndFastFieldWithDefer {
  ... on Query @defer {
    slowField
  }
  fastField
}

Then press the Play (Execute Query) button.

Alternatively, you can also send the defer operation via curl.

curl command for executing @defer operation

curl -g -X POST \
  -H "accept:multipart/mixed" \
  -H "content-type: application/json" \
  -d '{"query":"query SlowAndFastFieldWithDefer { ... on Query @defer { slowField } fastField }"}' \
  http://localhost:4000/graphql

Using Stream

The @stream directive allows you to stream the individual items of a field of the list type as the items are available.

Visit http://localhost:4000/graphql and paste the following operation into the left panel.

GraphQL Operation using @stream

query StreamAlphabet {
  alphabet @stream
}

Then press the Play (Execute Query) button.

Alternatively, you can also send the stream operation via curl.

curl -g -X POST \
  -H "accept:multipart/mixed" \
  -H "content-type: application/json" \
  -d '{"query":"query StreamAlphabet { alphabet @stream }"}' \
  http://localhost:4000/graphql

Writing safe stream resolvers

AsyncGenerators as declared via the async * keywords are prone to memory leaks and leaking timers. In the previous examples they are used for simplicity.

For real-world usage we recommend using Repeater as a safe alternative.

import { createYoga, createSchema, Repeater } from 'graphql-yoga'
import { createServer } from 'node:http'
import { useDeferStream } from '@graphql-yoga/plugin-defer-stream'
 
const yoga = createYoga({
  schema: createSchema({
    typeDefs: /* GraphQL */ `
      type Query {
        alphabet: [String!]!
      }
    `,
    resolvers: {
      Query: {
        alphabet: () =>
          new Repeater<string>(async (push, stop) => {
            const values = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
            const publish = () => {
              const value = values.shift()
              console.log('publish', value)
 
              if (value) {
                push(value)
              }
 
              if (values.length === 0) {
                stop()
              }
            }
 
            let interval = setInterval(publish, 1000)
            publish()
 
            await stop.then(() => {
              console.log('cancel')
              clearInterval(interval)
            })
          })
      }
    }
  }),
  plugins: [useDeferStream()]
})
 
const server = createServer(yoga)
 
server.listen(4000, () => {
  console.info('Server is running on http://localhost:4000/graphql')
})

If you execute the following GraphQL operation via GraphiQL and click the stop button before the last value has been sent to the client, no further values will be published as the underlying source (Repeater) has been stopped and disposed. In contrast, just using a AsyncGenerator would continue to run timers even after the client has stopped listening.

GraphQL Operation using @stream

query StreamAlphabet {
  alphabet @stream
}

Last updated on November 15, 2022

File Uploads Request Batching