Instrument a Service with Wide Events

Structured logging only pays off if the fields are actually there when an incident hits. This is the reference for wiring a wide event into a request handler correctly the first time, in whichever language the service happens to be written in.

In one line

Build one event object per request, add fields to it as the request progresses, and emit it exactly once at the end (success or error). Search by the highest-cardinality field you have (user ID, order ID, request ID) instead of grepping strings, and tail-sample once volume makes logging everything expensive.

You're here when

Signs this is your situation

A new service, job, or webhook handler and you're about to add its first logger.info calls
An incident review that ends in "the logs didn't have the field we needed"
A user-reported bug where searching their ID returns some, but not all, of the relevant lines
Concurrent requests interleave in the log viewer and you can't tell which line belongs to which one

The play

1
At the top of the handler, create one event object seeded with whatever you already know: a request ID, the route, the method.
2
Wrap the handler body so every exit path, success, an early return, an error, still reaches the emit step. Nothing else changes about the handler's control flow.
3
At each point you'd normally have written a step-by-step log line, set a field on the event instead. Set fields on both the happy path and the error path, not just on success.
4
Emit the event exactly once, with everything accumulated so far plus a duration: one structured-log call in a finally/defer, or, in a language with RAII, a span that logs itself on close.
5
Pick field names once and reuse them everywhere (userId, not user_id in one handler and uid in another). A field searched two different ways is two searches, not one.
6
Prefer the highest-cardinality identifier you have available at query time: a user ID or order ID narrows a search far more than a status code or route ever will.

@itsnas checkout.rs

codecheckout.rs

// The span's own fields *are* the wide event: declared empty here,
// filled in as the handler learns them, emitted once when the span
// closes (see main.rs).
#[tracing::instrument(
    name = "checkout",
    skip(state, req),
    fields(
        request_id = %uuid::Uuid::new_v4(),
        user_id = tracing::field::Empty,
        order_id = tracing::field::Empty,
        outcome = tracing::field::Empty,
    )
)]
async fn checkout(
    State(state): State<AppState>,
    Json(req): Json<CheckoutRequest>,
) -> Result<Json<CheckoutResponse>, AppError> {
    let span = tracing::Span::current();
 
    // Step 1: load the cart, record who this request is for the
    // moment we know it.
    let cart = state
        .db
        .load_cart(&req.cart_id)
        .await
        .inspect_err(|_| span.record("outcome", "error"))?;
    span.record("user_id", cart.user_id.to_string().as_str());
 
    // Step 2: place the order, record the outcome either way.
    let order = place_order(&state, &cart)
        .await
        .inspect_err(|_| span.record("outcome", "error"))?;
    span.record("order_id", order.id.to_string().as_str());
    span.record("outcome", "success");
 
    Ok(Json(CheckoutResponse { order_id: order.id }))
}

main

Nas (@itsnas)

async function checkout(req: Request) {
  const event = createEvent('checkout', { requestId: req.id })
 
  try {
    const cart = await loadCart(req.cartId)
    event.set({ userId: cart.userId })
 
    const order = await placeOrder(cart)
    event.set({ orderId: order.id, outcome: 'success' })
 
    return json(200, { orderId: order.id })
  } catch (error) {
    event.set({ outcome: 'error' })
    throw error
  } finally {
    event.emit()
  }
}

main

Nas (@itsnas)

func checkout(w http.ResponseWriter, r *http.Request) {
    fields := []any{"request_id", uuid.NewString()}
    outcome := "success"
 
    // A deferred call runs on every exit path, panics included, so
    // this is Go's own version of "emit from finally": nothing
    // downstream has to remember to call it.
    defer func() {
        slog.Info("checkout", append(fields, "outcome", outcome)...)
    }()
 
    cart, err := loadCart(r.Context(), cartID(r))
    if err != nil {
        outcome = "error"
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    fields = append(fields, "user_id", cart.UserID)
 
    order, err := placeOrder(r.Context(), cart)
    if err != nil {
        outcome = "error"
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    fields = append(fields, "order_id", order.ID)
 
    json.NewEncoder(w).Encode(order)
}

main

Nas (@itsnas)

async def checkout(request: Request) -> Response:
    log = structlog.get_logger().bind(request_id=str(uuid4()))
    outcome = "success"
 
    try:
        cart = await load_cart(request.cart_id)
        log = log.bind(user_id=cart.user_id)
 
        order = await place_order(cart)
        log = log.bind(order_id=order.id)
 
        return JSONResponse({"order_id": order.id})
    except Exception:
        outcome = "error"
        raise
    finally:
        log.bind(outcome=outcome).info("checkout")

main

Nas (@itsnas)

private static readonly ActivitySource Source = new("Checkout");
 
public async Task<IResult> Checkout(CheckoutRequest req, AppState state)
{
    // `using` disposes (ends) the activity on every exit path, the
    // same guarantee Rust's Drop and Go's defer give their own
    // versions of this pattern.
    using var activity = Source.StartActivity("checkout");
    activity?.SetTag("request_id", Guid.NewGuid());
 
    try
    {
        var cart = await state.Db.LoadCartAsync(req.CartId);
        activity?.SetTag("user_id", cart.UserId);
 
        var order = await PlaceOrderAsync(state, cart);
        activity?.SetTag("order_id", order.Id);
        activity?.SetTag("outcome", "success");
 
        return Results.Ok(new CheckoutResponse(order.Id));
    }
    catch
    {
        activity?.SetTag("outcome", "error");
        throw;
    }
}

main

Nas (@itsnas)

Same method, five ecosystems, and the emit step is where they actually diverge. #rust's #tracing span and C#'s System.Diagnostics.Activity both piggyback on the language's own guaranteed-cleanup primitive (Drop, using/IDisposable), so the emit happens on every exit path for free once it's wired up once. Go's defer earns the same guarantee by hand, one line at the top of the function. TypeScript and #python (via #structlog's bind()) have neither, so the try/finally in The play above is doing real work in those two, not just following convention; leave it out and an early return is a request with no event at all. The Wide Event Logger snip and Your Logging Sucks work the TypeScript version through in full.

Which path

New service, no logging yet

Skip step-by-step logging entirely, start with wide events from the first handler.
Standardize field names in one shared type before the second handler copies the first one's typos.

Existing service, already logging step-by-step

Retrofit the highest-traffic, most-incident-prone endpoints first; don't do a repo-wide sweep in one PR.
Leave the old log lines in place until the wide event has shipped and been used in at least one real debugging session, then remove them.

Gotchas

1
Logging inside a loop or a called helper
Reintroduces the interleaving problem one function at a time. Pass the event down (or use request-scoped context, or the ambient span a #[tracing::instrument]-style helper already gives you) and set the field from inside the helper instead of writing a fresh log line.
2
Only setting fields you need for the success case
The decline reason, the retry count, the partial state: almost everything worth knowing during an incident only exists on the failure path. Set it there, not just on success.
3
Excluding high-cardinality fields to save on ingest cost
User ID, order ID, and request ID are exactly the fields worth searching by. A column-oriented log store doesn't choke on them the way older string-search platforms did; dropping them to save money removes the fields that make the event useful.
4
Shipping this and leaving every call site logging step-by-step
The helper itself does nothing without call sites actually setting meaningful fields on it. Treat the rollout as rewriting the log calls, not just installing a library.
5
Logging every field at 100% once traffic is real
Wide events cost more bytes per request than a short line does. At volume, keep 100% of errors and slow requests, and tail-sample the boring successful ones down to a few percent.

Confirm you're clear

1
Pick a real incident from the last month and check whether the wide event for that request would have answered it in one query.
2
Two concurrent requests to the same endpoint produce two separate, fully-tagged rows, not interleaved fragments.
3
Field names match across every handler that emits them; a search by userId finds every service, not just the one you last edited.
4
A deliberately thrown error inside the handler still produces an emitted event with outcome: 'error'.

Go deeper

Your Logging Sucks Wide Event Logger