# Background Jobs

Run work outside the request that triggered it, with retries and a visible history.

Some work should not happen while a user waits: sending a batch of email, regenerating
images, calling a slow third-party API. A job is that work, queued and run later.

## Availability

|            |                                                                                 |
| ---------- | ------------------------------------------------------------------------------- |
| Built in   | No plugin required                                                              |
| Admin      | A **Background Jobs** screen showing queued, running, failed and completed work |
| Permission | Viewing jobs is an admin capability; a job runs as a user you nominate          |

## Define a job

```ts
import { defineJob } from "nextly";

export const sendDigest = defineJob({
  slug: "send-digest",
  handler: async (input, context) => {
    // `input` is the payload the job was enqueued with. `context` is what the
    // runner tells the handler about the run it is in.
  },
});
```

`defineJob` only builds the definition. Register it in your config, or nothing will run it:

```ts
import { defineConfig } from "nextly";

import { sendDigest } from "./jobs/send-digest";

export default defineConfig({
  jobs: [sendDigest],
});
```

`jobs` takes untyped definitions, so a job written with a type argument, `defineJob<{ ... }>`, is a type error there and fails `next build`. Leave the type argument off and check `input` at the top of the handler, as the samples below do. The payload comes from whoever queued the job, so checking it is worth doing anyway.

> [!WARNING]
> A job that is defined but not listed in `jobs` is absent from the registry. Rows queued for
> its slug sit there with no handler to pick them up.

## Retries and backoff

A failed job is retried with exponential backoff, starting at **1 second** and capped at
**1 hour**, with jitter so a batch of simultaneous failures does not retry in lockstep.

`attemptCount` counts the attempt that just failed, so a job with a budget of three
attempts gives up when the third fails — not after a fourth. That boundary is the usual
bug in retry code, so it is worth stating rather than leaving to be discovered.

## Who a job runs as

A job is not run with the authority of whoever enqueued it. It resolves a **run-as** user,
and that identity is what access control sees while the job runs, so a job cannot quietly
do more than the user it runs as is allowed to do.

## Running the queue

Queuing a job records it. Nothing runs it until something calls the jobs route, so a
Nextly installation with no scheduler has a queue that fills and never drains.

The scaffolded apps mount Nextly's handlers at `src/app/admin/api/[[...params]]/route.ts`,
so the URL is:

```
POST /admin/api/jobs/run
```

If you mounted the handlers somewhere else, use that path instead.

### Authenticating the caller

The route is not open. A scheduler has no session, so it authenticates with a shared
secret: set `NEXTLY_DRAIN_SECRET` (or `CRON_SECRET`) to a value of **at least 32
characters**, and send it as a bearer token.

```
POST /admin/api/jobs/run
Authorization: Bearer $NEXTLY_DRAIN_SECRET
```

The two variables fail differently when the value is too short, and knowing which one you
set decides where to look:

|                       | A value under 32 characters                                                                                                                                                 |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `NEXTLY_DRAIN_SECRET` | **The app refuses to boot.** The environment schema requires 32, so this fails validation before any request reaches the route                                              |
| `CRON_SECRET`         | The app starts and prints a warning at boot. The route refuses a `GET` (how Vercel Cron calls it) with a 403; a `POST` falls through to the permission check and gets a 401 |

`CRON_SECRET` is Vercel Cron's convention, which is why it is accepted at all and why it is
tolerated rather than rejected.

A signed-in human can pull the trigger instead, with the `manage-background-jobs`
permission and a POST. `GET` always requires the secret, because a permission check alone
would let a browser be steered into firing it.

> [!WARNING]
> Three ways to get a queue that grows and never drains, and the status code tells them
> apart:
>
> | Response | What happened                                                                                                                                  |
> | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
> | 404      | Pointed at `/api/jobs/run`; a scaffolded app has no general `/api` handler                                                                     |
> | 401      | No `Authorization` header, or a bearer the route does not accept, on a `POST`. It falls through to ordinary authentication and finds no caller |
> | 403      | A `GET` with no usable secret. That route never falls back to a permission check                                                               |
>
> Only the short-`CRON_SECRET` case announces itself, and it does so at boot rather than at
> request time. Check your scheduler's own log for the response code the first time you wire
> it up.

A pass is bounded. It claims at most 10 queued jobs, and it stops **starting** new ones
after 20 seconds. That 20 seconds is a budget for starting work, not a ceiling on the
request: the check happens before each claim, so a handler already running is awaited to
completion and the call can overrun. Size any serverless timeout for your slowest handler,
not for 20 seconds.

The queue is durable, so a pass that stops early loses no work: the next call continues
from the same table.

> [!WARNING]
> **Delivery is at-least-once, so handlers must be idempotent.** A lease stops two runners
> claiming a job at the same instant, and a fence stops a runner that lost its lease
> recording an outcome over its successor's. Neither stops a handler that OUTLIVES its
> lease: a process that stalls, is paused, or loses its connection stops renewing while its
> side effects are still in flight, and a later pass can reclaim and run the same job.
>
> The fence can refuse the first runner's database write. It cannot undo an email it already
> sent. This is the same contract SQS, BullMQ and pg-boss give, for the same reason.

### Writing a handler that can run twice

Every handler is told the id of the job it is running, and that id is the same on every
attempt at it. It is the key to hand anything the database cannot take back:

```ts
import { defineJob } from "nextly";

export const chargeInvoice = defineJob({
  slug: "charge-invoice",
  async handler(input, { jobId }) {
    const invoiceId = typeof input?.invoiceId === "string" ? input.invoiceId : null;
    const cents = typeof input?.cents === "number" ? input.cents : null;
    if (invoiceId === null || cents === null) {
      throw new Error("charge-invoice needs { invoiceId: string, cents: number }");
    }

    // The provider de-duplicates on this key, so a second run of the same job
    // returns the first charge instead of making a second one.
    const response = await fetch("https://api.payments.example/charges", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Idempotency-Key": jobId,
      },
      body: JSON.stringify({ invoiceId, amountCents: cents }),
    });

    // `fetch` resolves for a 429 or a 500. Returning here would finish the job
    // as done and the charge would never be made. Throw, so the runner records
    // a failed attempt and the backoff policy retries it.
    if (!response.ok) {
      throw new Error(
        `charge for ${invoiceId} failed: ${String(response.status)}`
      );
    }
  },
});
```

> [!WARNING]
> **`fetch` only rejects on a network failure.** An HTTP 4xx or 5xx resolves normally, so a
> handler that does not check `response.ok` returns, and the runner marks the job **done**.
> The queue believes the work happened. Throw on a response you did not want, and let the
> retry policy do its job.

Where to put the key depends on the provider. Stripe and Adyen read an `Idempotency-Key`
request header; Square takes an `idempotency_key` field in the request body. Check your
provider's own documentation rather than assuming the header. Where the side effect is a
row rather than a call, the same idea is an upsert against a unique column holding the key,
so the second attempt collides instead of inserting.

**One key per side effect, not one key per job.** A handler that charges a card and then
posts to a ledger must not send the same `jobId` to both if they share a uniqueness
namespace, or the second is rejected as a duplicate of the first. Derive one key per
operation:

```ts
import { defineJob } from "nextly";

export const settleInvoice = defineJob({
  slug: "settle-invoice",
  async handler(input, { jobId }) {
    const invoiceId = typeof input?.invoiceId === "string" ? input.invoiceId : null;
    if (invoiceId === null) throw new Error("settle-invoice needs { invoiceId: string }");

    await charge(`${jobId}:charge`, invoiceId);
    await postToLedger(`${jobId}:ledger`, invoiceId);
  },
});

declare function charge(key: string, invoiceId: string): Promise<void>;
declare function postToLedger(key: string, invoiceId: string): Promise<void>;
```

Deriving the key from the input instead works only while the input is unique, and that is
a property of whoever queued the job rather than of the queue. The same payload queued
twice is two jobs with two ids, and both are meant to happen.

The context also carries `attempt`, counting from 1. It is written to the row before the
handler starts, so a handler that dies part-way still leaves the count advanced and the
next run is numbered higher. Use it for behaviour that should change on a retry, such as
logging more loudly, or giving up on an optimisation that failed last time.

It still does not tell you whether a side effect landed. `attempt > 1` says an earlier run
began, not what it finished, and nothing inside this process can know what a provider did
with a request it never answered. That is the question `jobId` answers by asking the
provider.

### The lease

The lease is renewed for you every third of its length while a handler is alive, so
ordinary long work does not expire it. What it bounds is a runner that stops renewing: a
process that stalls, is paused, or loses its connection.

The built-in route uses the fixed 30 second default and takes no configuration. A project
running its own pass can call `runJobsPass` directly and pass `leaseMs`, which is worth
raising where an event loop pause or a slow database can plausibly exceed 30 seconds,
because the renewal timer cannot fire during a stall either.

## Retention

Finished jobs are pruned on a schedule, so the table holds recent history rather than
everything that has ever run.

## Related

* [Webhooks](https://nextlyhq.com/docs/webhooks.md) — telling another system that something changed
* [Releases](https://nextlyhq.com/docs/releases.md) — scheduling one depends on this queue draining
* [Audit log](https://nextlyhq.com/docs/audit-log.md)