# ADR: Telemetry that outlives the container

Status: accepted, 2026-09-03, shipped as 1.0.0.34. Steve asked for
Azure error handling shown on the admin page, with every API call, every API
error and every React error logged, in Application Insights if the solo
subscription includes it. It does: the free trial includes it, and the first 5 GB a month
of ingestion costs nothing.

## In plain words

This page sends the site's requests and errors to a store in Azure that keeps them when the container is replaced (Application Insights), on the free tier with a daily cap so it cannot generate a charge. The Admin tab reads the last hour back from it, and the key that sends the data never enters the repository.

What that is worth: a developer can see which route is slowest or what threw an hour ago, and the organization gets lasting monitoring at no cost that never blocks a deploy.

## Context

ADR: Observability built the Admin tab on an in-memory ring buffer of fifty
entries, and said out loud what that costs: the buffer resets on every roll,
holds nothing older than the current container, and cannot answer a question
about last Tuesday. ADR: Error handling then routed browser errors into the
same buffer. That made it more useful and no more durable.

Structured request logs go to the container's stdout, where Azure keeps
them for a container group's lifetime and nobody reads them. Neither the
logs nor the buffer can answer the questions worth asking about a running
site: how many requests failed in the last hour, which route is slowest,
what threw and when.

## Decision

**Application Insights, in the same resource group, on the free tier.** A
Log Analytics workspace (`log-theyard-ss`, PerGB2018, 30-day retention) and
a component bound to it (`appi-theyard-ss`), both in RG-THEYARD-SS. The
daily ingestion cap is set to 0.1 GB with collection stopped at the cap, so
the resource cannot generate a charge even if something goes wrong.

**The API sends with the Azure Monitor OpenTelemetry distro.** One call in
Program.cs gives requests, dependencies and exceptions with their durations,
correlated by trace id. It is registered only when a connection string is
present. So a local run and every test are untouched and need no fake.

**The connection string never enters the repository.** It is an ingestion
key. The deploy workflow reads it from Azure at roll time with the federated
credential it already uses, masks it in the log, and substitutes it into the
container spec, which carries a `__APPINSIGHTS_CONNECTION_STRING__`
placeholder in the repository. The app treats any value starting with `__`
as absent, so a manual rollback with the committed file runs clean instead
of sending telemetry to nowhere.

That read is allowed to fail. If Azure does not answer, the roll leaves the
placeholder in place and logs a workflow warning, then ships anyway. Telemetry is
an addition to this system, not a dependency of it, and a delivery pipeline
that a monitoring resource can block is a worse trade than an hour of
missing traces.

**Browser errors go through the API, not straight to Azure.** The boundary
and the two window handlers already POST to `/api/errors/client`
(ADR: Error handling); that endpoint now also writes a structured log, which
the distro forwards. Three things follow: the page loads no second external
script, the ingestion key stays server-side, and a browser error is
searchable beside the server's own with the same trace correlation.

**The Admin tab reads it back with the container's own identity.**
`id-theyard-ss` was granted Monitoring Reader on the component, so the
container asks Application Insights about itself the same way it already
asks Azure Resource Manager about its own container group. No key is stored
for reading either.

## In the code

The registration, and the reader it hands the Admin tab
(`api/TheYard.Api/Program.cs`):

```csharp
// Application Insights. The connection string is an ingestion key, so it is
// never in the repository: the deploy reads it from Azure at roll time and
// passes it to the container as an environment variable. Absent, as it is
// locally and in every test, this block does nothing and the app runs without
// telemetry, which is why no test needs a fake for it.
string? telemetryConnection = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
bool telemetryOn = !string.IsNullOrWhiteSpace(telemetryConnection)
    && !telemetryConnection.StartsWith("__", StringComparison.Ordinal);
if (telemetryOn)
{
    builder.Services.AddOpenTelemetry().UseAzureMonitor(options =>
    {
        options.ConnectionString = telemetryConnection;
    });
}
// The component's app id is public (it is not a key) and is what the Admin
// tab queries with the container's managed identity.
var telemetry = new TelemetryReader(
    builder.Configuration["Azure:AppInsightsAppId"] ?? "6ff89351-7fcc-4a41-8238-db65c5903c36",
    YardComposition.AzureClientId(builder.Configuration),
    // Wired only where the connection string is: the app id has a default and
    // is therefore no evidence at all that this build can read anything.
    enabled: telemetryOn);
builder.Services.AddSingleton(telemetry);
```
*Live from [`api/TheYard.Api/Composition/TelemetryRegistration.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Composition/TelemetryRegistration.cs#L15-L38), region `telemetry`, read from this build at 1d6b50b.*

```csharp
// The last hour as Application Insights has it, for the Admin tab (ADR-024).
// Answers a shape the card can render even when telemetry is off or the query
// fails, because a panel that reports on the system must not be able to break
// the page it reports from.
app.MapGet("/api/admin/telemetry", Telemetry);
```
*Live from [`api/TheYard.Api/Endpoints/AdminEndpoints.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Endpoints/AdminEndpoints.cs#L221-L225), region `telemetry-endpoint`, read from this build at 1d6b50b.*

One query answers the whole card, because three questions in three round
trips is three chances to time out (`api/TheYard.Api/Telemetry.cs`):

```csharp
/// <summary>
/// One query, five answers, so the Admin card costs a single round trip:
/// how the last hour's requests went, the slowest routes, the exceptions,
/// how many browser errors arrived, and the newest request of the last day.
/// Several questions in several round trips would be several chances to
/// time out.
///
/// Three things here are not stylistic. `part` labels each block because
/// `kind` is a reserved word and a query using it does not parse. The
/// labels sit in `extend` rather than `summarize`, because summarize takes
/// aggregations only. And `success` is compared as text because the classic
/// Application Insights schema stores it as the string "True" while the
/// workspace schema stores a bool; comparing it to a bool is a 400 in one
/// of the two (ADR: Telemetry that outlives the container, second pass).
///
/// Every table is filtered to this site's cloud_RoleName when the site's
/// name is known, because both sites send to the same component; `site_role` is
/// empty when it is not known, and the filter then lets every row through.
/// Counts are sum(itemCount) rather than count(): the distro samples traces
/// by default, and a kept row stands for itemCount requests.
///
/// The newest request of the last day is how the card tells an empty hour
/// from a quiet site: a component at its daily data cap takes nothing more
/// until midnight UTC, and the card would otherwise read that as no traffic.
/// `newest_at` and not `last`: `last` does not parse.
/// </summary>
private const string Query = """
    let lookback = 1h;
    let site_role = "__SITE__";
    let summary = requests
        | where timestamp > ago(lookback)
        | where isempty(site_role) or cloud_RoleName =~ site_role
        | summarize total = sum(itemCount), failed = sumif(itemCount, tostring(success) !in ("True", "true")),
                    p50 = percentile(duration, 50), p95 = percentile(duration, 95)
        | extend part = "requests", p50 = round(p50, 1), p95 = round(p95, 1)
        | project part, total, failed, p50, p95;
    let slowest = requests
        | where timestamp > ago(lookback)
        | where isempty(site_role) or cloud_RoleName =~ site_role
        | summarize calls = sum(itemCount), avg_ms = avg(duration) by route = name
        | top 5 by avg_ms desc
        | extend part = "slowest", avg_ms = round(avg_ms, 1)
        | project part, route, calls, avg_ms;
    let failures = exceptions
        | where timestamp > ago(lookback)
        | where isempty(site_role) or cloud_RoleName =~ site_role
        | extend err_type = tostring(type), err_method = tostring(method)
        | summarize hits = sum(itemCount), last_at = max(timestamp) by err_type, err_method
        | top 5 by last_at desc
        | extend part = "exception"
        | project part, err_type, err_method, hits, last_at;
    let browser = traces
        | where timestamp > ago(lookback)
        | where isempty(site_role) or cloud_RoleName =~ site_role
        | where message startswith "Browser error on"
        | summarize hits = sum(itemCount), last_at = max(timestamp)
        | extend part = "browser"
        | project part, hits, last_at;
    let newest = requests
        | where timestamp > ago(1d)
        | where isempty(site_role) or cloud_RoleName =~ site_role
        | summarize newest_at = max(timestamp)
        | extend part = "newest"
        | project part, newest_at;
    union summary, slowest, failures, browser, newest
    """;

/// <summary>The query with this site's name in place of the marker; empty when the name is not known.</summary>
private static string QueryFor(string? siteName) => Query.Replace("__SITE__", siteName ?? "", StringComparison.Ordinal);

/// <summary>
/// An App Service site name is letters, digits and hyphens. Anything else
/// is not a site name, and it is never put into a query: the value goes
/// inside a Kusto string literal, so a quote in it would end the literal.
/// </summary>
private static string? SiteNameOrNull(string? name) =>
    !string.IsNullOrWhiteSpace(name) && name.Length <= 60 && name.All(c => char.IsAsciiLetterOrDigit(c) || c == '-')
        ? name
        : null;

/// <summary>
/// Which requests the card's figures count, in words: this site alone when
/// its name is known, or every site that reports to the component when it
/// is not, so a figure from both sites is never read as one site's.
/// </summary>
private static string ScopeFor(string? siteName) => siteName is null
    ? "Counts every site that reports to this Application Insights component: both sites share it, so these figures are both sites together."
    : $"Counts {siteName} only. The other site reports to the same Application Insights component and is left out.";
```
*Live from [`api/TheYard.Api/Telemetry.Kql.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Telemetry.Kql.cs#L11-L98), region `kql`, read from this build at 1d6b50b.*

```csharp
/// <summary>
/// The last hour, as the Admin tab shows it. Every failure answers with a
/// shape the card can render rather than throwing: a telemetry panel that
/// breaks the page it reports on would be worse than useless (ADR-010).
/// The failure note carries what the service actually said, because a
/// status code alone does not name the cause, and finding it otherwise
/// takes a deploy.
/// </summary>
public async Task<object> GetRecentAsync()
{
    if (!Configured)
    {
        return new
        {
            configured = false,
            note = "Application Insights is not configured for this build. It is wired at deploy time from Azure; a local run reports nothing.",
        };
    }
    lock (_gate)
    {
        // A minute of cache: the Admin tab is a page someone leaves open,
        // and the query costs a token exchange plus a round trip.
        if (_cached is not null && DateTimeOffset.UtcNow - _cachedAt < TimeSpan.FromSeconds(60))
        {
            return _cached;
        }
    }
    try
    {
        // Whichever door this host has: the metadata address on Container
        // Instances, the named endpoint on App Service (ADR: One plan, two sites).
        string token = await IdentityTokens.AcquireAsync(Http, "https://api.applicationinsights.io", clientId);

        using var req = new HttpRequestMessage(HttpMethod.Post,
            $"https://api.applicationinsights.io/v1/apps/{appId}/query")
        {
            Content = JsonContent.Create(new { query = QueryFor(_siteName) }),
        };
        req.Headers.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
        using var resp = await Http.SendAsync(req);
        string payload = await resp.Content.ReadAsStringAsync();
        if (!resp.IsSuccessStatusCode)
        {
            return Cache(Failed($"the telemetry API answered {(int)resp.StatusCode}: {Trim(payload)}"));
        }
        using var body = JsonDocument.Parse(payload);
        return Cache(Shape(body, _siteName));
    }
    catch (Exception ex)
    {
        return Cache(Failed(ex.GetType().Name));
    }
}
```
*Live from [`api/TheYard.Api/Telemetry.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Telemetry.cs#L52-L104), region `read`, read from this build at 1d6b50b.*

Kusto answers in columns and rows; the card wants objects. Reading each row
by column name rather than position is what keeps a query edit from shifting
every value silently:

```csharp
/// <summary>
/// Kusto answers with columns and rows, not objects. This turns the one
/// table into the pieces the card renders, reading each row by the
/// column's name rather than its position, because a query edit that adds
/// a column would otherwise shift every value silently. A union pads the
/// columns a row does not have, so each branch reads only its own. The
/// site and the scope sentence say whose requests the figures are.
/// </summary>
private static object Shape(JsonDocument body, string? siteName)
{
    var table = body.RootElement.GetProperty("tables")[0];
    var columns = table.GetProperty("columns").EnumerateArray()
        .Select((c, i) => (Name: c.GetProperty("name").GetString() ?? "", Index: i))
        .ToDictionary(c => c.Name, c => c.Index, StringComparer.Ordinal);

    string? Text(JsonElement row, string column) =>
        columns.TryGetValue(column, out int i) && row[i].ValueKind is not JsonValueKind.Null
            ? row[i].ToString()
            : null;

    double? Number(JsonElement row, string column) =>
        columns.TryGetValue(column, out int i) && row[i].ValueKind is JsonValueKind.Number
            ? row[i].GetDouble()
            : null;

    object? summary = null;
    object? browser = null;
    string? newest = null;
    var slowest = new List<object>();
    var exceptions = new List<object>();

    foreach (var row in table.GetProperty("rows").EnumerateArray())
    {
        switch (Text(row, "part"))
        {
            case "requests":
                summary = new
                {
                    total = (int)(Number(row, "total") ?? 0),
                    failed = (int)(Number(row, "failed") ?? 0),
                    p50_ms = Number(row, "p50"),
                    p95_ms = Number(row, "p95"),
                };
                break;
            case "slowest":
                slowest.Add(new
                {
                    name = Text(row, "route") ?? "",
                    calls = (int)(Number(row, "calls") ?? 0),
                    avg_ms = Number(row, "avg_ms"),
                });
                break;
            case "exception":
                exceptions.Add(new
                {
                    type = Text(row, "err_type") ?? "",
                    method = Text(row, "err_method") ?? "",
                    count = (int)(Number(row, "hits") ?? 0),
                    last_at = Text(row, "last_at") ?? "",
                });
                break;
            case "newest":
                newest = Text(row, "newest_at");
                break;
            case "browser":
                browser = new
                {
                    count = (int)(Number(row, "hits") ?? 0),
                    last_at = Text(row, "last_at") ?? "",
                };
                break;
        }
    }

    return new
    {
        configured = true,
        available = true,
        window = "the last hour",
        site = siteName,
        scope = ScopeFor(siteName),
        summary = summary ?? new { total = 0, failed = 0, p50_ms = (double?)null, p95_ms = (double?)null },
        slowest,
        exceptions,
        browser = browser ?? new { count = 0, last_at = "" },
        newest_request_at = newest,
    };
}
```
*Live from [`api/TheYard.Api/Telemetry.Shape.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Telemetry.Shape.cs#L13-L100), region `shape`, read from this build at 1d6b50b.*

The card, which renders every state the reader can answer with
(`src/components/admin/TelemetryCard/TelemetryCard.tsx`, a chunk of its own since the workbench):

```tsx
{/* Application Insights, read back through the container's own identity
            (ADR-024). Every state the reader can answer with is rendered here:
            not configured (a local run), configured but unreadable, and the
            happy path. A telemetry panel that can break the page it reports on
            would be worse than no panel. */}
<article className={`${styles.card} op-glass`} data-testid="telemetry-card">
  <h2 className={styles.cardTitle}>Traffic, last hour</h2>
  {telemetry === null ? (
    <CardState state="loading" />
  ) : telemetry === 'failed' ? (
    failed('the telemetry')
  ) : !telemetry.configured || telemetry.available === false ? (
    <p className={styles.muted} data-testid="telemetry-note">
      {telemetry.note}
    </p>
  ) : emptyHour !== null ? (
    <>
      {scopeLine(telemetry)}
      <p className={styles.muted} data-testid="telemetry-empty">
        {emptyHour}
      </p>
    </>
  ) : (
    <>
      {scopeLine(telemetry)}
      <div className={styles.statusRow}>
        <span className={pill((telemetry.summary?.failed ?? 0) === 0)}>
          {telemetry.summary?.total ?? 0} request
          {(telemetry.summary?.total ?? 0) === 1 ? '' : 's'}
        </span>
        <span className={styles.muted}>{telemetry.summary?.failed ?? 0} failed</span>
        <span className={styles.mono}>p50 {reading(telemetry.summary?.p50_ms)}</span>
        <span className={styles.mono}>p95 {reading(telemetry.summary?.p95_ms)}</span>
        {/* Every React error counts, so the count of them is on the card
                    rather than only in the portal. */}
        <span className={pill((telemetry.browser?.count ?? 0) === 0)}>
          {telemetry.browser?.count ?? 0} browser
        </span>
      </div>
      {telemetry.slowest && telemetry.slowest.length > 0 && (
        <p className={styles.muted}>Slowest routes, by average time</p>
      )}
      <ul className={styles.checkList}>
        {telemetry.slowest?.map((route) => (
          <li key={route.name} className={styles.checkRow} data-testid="telemetry-route">
            <span className={styles.mono}>{route.name}</span>
            <span className={styles.duration}>avg {reading(route.avg_ms)}</span>
            <span className={styles.muted}>
              {route.calls} call{route.calls === 1 ? '' : 's'}
            </span>
          </li>
        ))}
      </ul>
      {telemetry.exceptions && telemetry.exceptions.length > 0 && (
        <>
          <p className={styles.muted}>Exceptions</p>
          <ul className={styles.errorList}>
            {telemetry.exceptions.map((entry, index) => (
              <li key={index} className={styles.errorRow} data-testid="telemetry-exception">
                <span className={styles.mono}>{entry.type}</span>
                <span className={styles.muted}>{entry.method}</span>
                <span className={styles.muted}>
                  {entry.count} time{entry.count === 1 ? '' : 's'}
                </span>
              </li>
            ))}
          </ul>
        </>
      )}
    </>
  )}
</article>
```
*Live from [`src/components/admin/TelemetryCard/TelemetryCard.tsx`](https://github.com/SteveStout/TheYard/blob/1d6b50b/src/components/admin/TelemetryCard/TelemetryCard.tsx#L48-L119), region `telemetry-card`, read from this build at 1d6b50b.*

The container spec's placeholder, and the roll step that fills it in
(`infra/aci-theyard.yaml`, `.github/workflows/deploy.yml`):

```yaml
# The image line is a placeholder the pipeline rewrites per roll; everything
# else is the shape of the container group. This file is the rollback path
# too: `az container create -f` with it restores a known-good group.
containers:
- name: theyard
  properties:
    configMap:
      keyValuePairs: {}
    environmentVariables:
    # Filled in at roll time by the deploy workflow, which reads the value
    # from Azure with the credential it already has (ADR-024). The literal
    # placeholder below is what a manual rollback would set, and the app
    # treats anything starting with __ as "no telemetry", so a rollback runs
    # clean rather than sending to nowhere.
    - name: APPLICATIONINSIGHTS_CONNECTION_STRING
      secureValue: __APPINSIGHTS_CONNECTION_STRING__
    # Azure SQL Database, substituted the same way (ADR: The SQL Server
    # backend). What lands here is a server, a database and
    # `Authentication=Active Directory Managed Identity`: no password, because
    # the server has no SQL login to have one. It is still written as a
    # secureValue, because the setting is where a secret would go if the
    # authentication model ever changed, and a placeholder left behind by a
    # failed substitution means the container falls back to SQLite rather than
    # crash-looping.
    - name: ConnectionStrings__YardSql
      secureValue: __YARD_SQL_CONNECTION__
    # The session signing key, filled at roll time from the repository secret
    # of the same name, so both containers sign with one key and a roll does
    # not end every session (ADR: Three readers with no memory of the
    # project). The only secret in the pipeline, and it never appears in a
    # log: GitHub masks it, and the roll substitutes it in python. A
    # placeholder left by a missing secret reads as "no key": the process
    # invents one at startup and logs that it did, which was every roll's
    # behaviour until 1.0.0.111.
    - name: Auth__SigningKey
      secureValue: __YARD_AUTH_SIGNING_KEY__
    # The operator's key for the Admin tab's visitor rows, filled at roll
    # time from the repository secret ADMIN_KEY the same way. A placeholder
    # left by a missing secret reads as "no key", and the endpoint the key
    # guards answers 404 to everybody until there is one (ADR: Site
    # activity, and the line an address does not cross).
    - name: Admin__Key
      secureValue: __ADMIN_KEY__
    # Azure Cosmos DB, beside the SQL setting rather than instead of it: this
    # container runs both stores and a visitor picks one with the toggle at
    # the top of the page (ADR: One container, both stores). An endpoint is a
    # URL, the database is a name, and the credential is the container's own
    # managed identity, so all three are plain values: there is no key on the
    # account to have leaked. The default names the store a request gets when
    # it names none, which on the live site is the relational one.
    - name: Cosmos__AccountEndpoint
      value: https://cosmos-theyard-ss.documents.azure.com:443/
    - name: Cosmos__Database
      value: theyard
    - name: Cosmos__Credential
      value: managed-identity
    - name: Store__Default
      value: sql
    # The one email this site sends, the password reset link, through Azure
    # Communication Services as the containers' own identity: an endpoint
    # and a sender address on its Azure-managed domain, both plain values,
    # no key (ADR: Accounts and per-user bids, addendum). Unset, the
    # "Forgot password" button says so and the operator's link is the way.
    - name: Email__Endpoint
      value: https://acs-theyard-ss.unitedstates.communication.azure.com
    - name: Email__From
      value: DoNotReply@ae4d5c47-28c9-49e2-8d3a-a859abd0f7df.azurecomm.net
    # Warm the other store right after the default one is serving, so the
    # first visitor to toggle does not pay for its catalogue. Off outside a
    # deploy, where ten test applications at once would each warm two.
    - name: Store__WarmOthers
      value: "true"
    # The other container, for the Admin tab's comparison card (ADR: Backends,
    # side by side). A public origin, read server side with a short patience,
    # and a plain value because a URL is not a secret.
    - name: Peer__Url
      value: http://theyard-cosmos-ss-zmnetj67bn5h2.westus2.azurecontainer.io:8080
    # The same container as a visitor reaches it, for the Store bar's link
    # to the other site: its own name under the domain, behind the same edge
    # since 1.0.0.100 (ADR: A permanent address for the second site).
    - name: Peer__Site
      value: https://theyard-cosmos.stevenstout.biz
    # This site as a visitor reaches it, for the links the site writes into an
    # email: behind the edge the request's own host is the origin, which is
    # not an address anybody should be sent (ADR: Accounts and per-user
    # bids, addendum of 14 September).
    - name: Site__Url
      value: https://theyard.stevenstout.biz
    image: __IMAGE__
    ports:
    - port: 8080
    resources:
      requests:
        cpu: 1.0
        memoryInGB: 1.5
```
*Live from [`infra/aci-theyard.yaml`](https://github.com/SteveStout/TheYard/blob/1d6b50b/infra/aci-theyard.yaml#L20-L114), region `container`, read from this build at 1d6b50b.*

```yaml
# A web app's settings live on the web app, so a roll is two calls: the
# four values the template does not hold, then the image
# (ADR: One plan, two sites). Both answer with -o none, because the first
# would otherwise print every setting the site holds into this log. The
# plan, the sites and every other setting are infra/appservice.bicep, and
# a roll changes none of them.
- name: Roll the site
  env:
    YARD_AUTH_SIGNING_KEY: ${{ secrets.YARD_AUTH_SIGNING_KEY }}
    ADMIN_KEY: ${{ secrets.ADMIN_KEY }}
  run: |
    # The Application Insights connection string is an ingestion key, so it
    # lives only in Azure: read it here with the credential this job already
    # holds and mask it in the log. az resource show is core CLI; az monitor
    # app-insights would pull an extension at run time, which is a second
    # thing that can fail a deploy. Telemetry is an addition, not a
    # dependency: if the read fails, the site keeps the value it has and
    # still ships, because a deploy pipeline held hostage to its own
    # observability is worse than a quiet hour of no traces.
    CONN=$(az resource show -g "$RG" -n "$AI_NAME" \
      --resource-type "Microsoft.Insights/components" \
      --query "properties.ConnectionString" -o tsv 2>/dev/null || true)
    if [ -n "$CONN" ]; then
      echo "::add-mask::$CONN"
    fi
    # The database. The value names a server, a database and an
    # authentication mode; the credential is the site's managed identity,
    # which never leaves Azure. The name is composed rather than read back
    # from Azure: the first version of this asked `az sql server show`,
    # which this job's identity is not allowed to do, the call failed
    # silently, and 1.0.0.49 shipped on SQLite and said so. An Azure SQL
    # logical server's address is its name plus the service suffix. No
    # Connect Timeout, on purpose: YardConnection widens a timeout nobody
    # gave to sixty seconds, and a value given here wins over that
    # (ADR: The SQL Server backend).
    YARD_SQL=""
    if [ -n "$SQL_SERVER" ]; then
      YARD_SQL="Server=tcp:${SQL_SERVER}.database.windows.net,1433;Initial Catalog=${SQL_DB};Encrypt=True;TrustServerCertificate=False;Authentication=Active Directory Managed Identity;User Id=${MI_CLIENT_ID};"
    fi
    # #region secrets
    # The session signing key and the operator's key, from the repository
    # secrets of the same names, handed to the step through env and written
    # to a file by python, so neither is ever part of a command line and
    # neither appears in a log: GitHub masks them, and the settings call
    # prints nothing. A value this job does not hold is left out, and the
    # site keeps what it has: the template's placeholder on a new site,
    # which the app reads as "no key", or the last roll's value
    # (ADR: Three readers with no memory of the project; ADR: Site
    # activity, and the line an address does not cross).
    CONN="$CONN" YARD_SQL="$YARD_SQL" python3 - "$RUNNER_TEMP/settings.json" <<'PY'
    import json, os, sys
    wanted = {
        "APPLICATIONINSIGHTS_CONNECTION_STRING": os.environ.get("CONN", ""),
        "ConnectionStrings__YardSql": os.environ.get("YARD_SQL", ""),
        "Auth__SigningKey": os.environ.get("YARD_AUTH_SIGNING_KEY", ""),
        "Admin__Key": os.environ.get("ADMIN_KEY", ""),
    }
    held = [{"name": name, "value": value, "slotSetting": False} for name, value in wanted.items() if value]
    for name, value in wanted.items():
        if not value:
            print(f"::warning::{name} is not held by this job; the site keeps the value it has")
    json.dump(held, open(sys.argv[1], "w", encoding="utf-8"))
    print(f"{len(held)} of {len(wanted)} values go to the site (values masked)")
    PY
    # #endregion secrets
    if [ "$(cat "$RUNNER_TEMP/settings.json")" != "[]" ]; then
      az webapp config appsettings set -g "$RG" -n "$APP" --settings "@$RUNNER_TEMP/settings.json" -o none
    fi
    rm -f "$RUNNER_TEMP/settings.json"
    # The image is the only thing that changes between rolls. App Service
    # starts the new container beside the old one and moves traffic when
    # the new one answers, so the site serves through its own roll.
    az webapp config set -g "$RG" -n "$APP" --linux-fx-version "DOCKER|$IMAGE" -o none
    az webapp show -g "$RG" -n "$APP" --query "{state:state, image:siteConfig.linuxFxVersion, host:defaultHostName}" -o json
```
*Live from [`.github/workflows/deploy.yml`](https://github.com/SteveStout/TheYard/blob/1d6b50b/.github/workflows/deploy.yml#L148-L221), region `roll`, read from this build at 1d6b50b.*

## What this cost, exactly

Two resources and one role assignment, all inside RG-THEYARD-SS, all
additive. Nothing else in the group changed. Steve's authorization is
recorded in the mentor notes with the scope it covers and the rollback,
which is `az monitor app-insights component delete` and
`az monitor log-analytics workspace delete`: the app runs unchanged without
the connection string.

## Consequences

- The Admin tab now shows the last hour from a store that survives a roll,
  beside the in-memory buffer that does not. Both are on the page on
  purpose: one is durable and a minute stale, the other is immediate.
- A trace id ties a ProblemDetails response (ADR: Error handling) to a
  request in the portal.
- Telemetry is off in development and in CI, which is the right default and
  also means the tests prove the off path rather than the on path. The on
  path is proved from the domain after each deploy.
- A deploy can now ship with telemetry off, and says so in the workflow log
  and on the Admin card. That is the intended failure, not a gap.
- The reader caches for a minute. The Admin tab refreshes every thirty
  seconds and would otherwise ask Azure twice a minute forever for a page
  someone left open.
- 30-day retention and a 0.1 GB daily cap are demo settings, not production
  ones. A real deployment would size both to its traffic and its audit
  needs, and would not put the Admin tab on a public route at all
  (ADR: Observability says why this one is).

## Where it sits

Telemetry is a host concern and stays in the host Api, where Composition/TelemetryRegistration.cs adds the exporter only when a connection string is present and TelemetryReader in Telemetry.cs reads the last hour back for the Admin tab's TelemetryCard in the front end. Application and Domain never learn that Application Insights exists, which is the onion's dependency rule at work, and no SOLID principle is in play beyond that. It cost two Azure resources held to a 0.1 GB daily cap and one Monitoring Reader grant, plus a deploy that can ship with telemetry switched off. A site with real traffic would size retention and the cap to its load and take these readings off a public route.

## Files

- [`api/TheYard.Api/Telemetry.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Telemetry.cs): the reader, its query and its shaping.
- [`api/TheYard.Api/Composition/TelemetryRegistration.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Composition/TelemetryRegistration.cs): the registration.
- [`api/TheYard.Api/Endpoints/ErrorEndpoints.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Endpoints/ErrorEndpoints.cs): the browser-error log.
- [`api/TheYard.Api/Endpoints/AdminEndpoints.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Endpoints/AdminEndpoints.cs): the Admin endpoint (region telemetry-endpoint).
- [`api/TheYard.Api/TheYard.Api.csproj`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/TheYard.Api.csproj): the one package this added.
- [`src/components/admin/TelemetryCard/TelemetryCard.tsx`](https://github.com/SteveStout/TheYard/blob/main/src/components/admin/TelemetryCard/TelemetryCard.tsx): the card and its three states.
- [`infra/aci-theyard.yaml`](https://github.com/SteveStout/TheYard/blob/main/infra/aci-theyard.yaml) and [`.github/workflows/deploy.yml`](https://github.com/SteveStout/TheYard/blob/main/.github/workflows/deploy.yml): the placeholder and the roll-time substitution.
- [`api/TheYard.Tests/TelemetryTests.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Tests/TelemetryTests.cs): the off path, the shape of every answer, and the endpoint.
- [`tests/e2e/admin.spec.ts`](https://github.com/SteveStout/TheYard/blob/main/tests/e2e/admin.spec.ts): the card rendering its not-configured state in a browser.
- [`docs/decisions/ADR-010-observability.md`](https://github.com/SteveStout/TheYard/blob/main/docs/decisions/ADR-010-observability.md) and [`docs/decisions/ADR-023-error-handling.md`](https://github.com/SteveStout/TheYard/blob/main/docs/decisions/ADR-023-error-handling.md): the Admin tab and the error path this extends.