# ADR: Site activity, and the line an address does not cross Status: accepted, 2026-09-13, shipped as 1.0.0.114. Steve asked for overall site activity on the Admin tab, by IP address and by store, Cosmos DB or SQL, with a graph of activity at the top, and ruled that user email addresses are private and never shown. ## In plain words The Admin tab now shows a graph of site activity, split by which database served each visit (Azure SQL or Azure Cosmos DB), kept in durable rows that survive a restart. No email address is ever recorded, and a visitor's address is kept only as a daily keyed hash plus its network (the first three octets). What that is worth: a developer sees a privacy rule kept by a data shape that has no field for the private thing, and the organization can publish its traffic on a public page without exposing who visited. ## Context The Admin tab could say what the container had served in the last few hundred requests, from a ring of five hundred entries that empties on every roll (ADR: Observability). It could not say what the site had served this week, or who had been by, or which of the two stores each visitor landed on. The telemetry that outlives the container is Application Insights on the free tier, thirty days at a tenth of a gigabyte a day (ADR: Telemetry that outlives the container), and it does not know which store served a request, because the store is decided by a header the request carries and the telemetry never asked. Two things shaped the answer before a line was written. **The Admin tab is public.** `GET /api/admin/metrics` answers 200 to anybody, and that is by design: the page is the running system reporting on itself, and the reasoning is on the Best Practices page. So "site activity by visitor" is not a private log the owner reads. It is a page any stranger can read, and a network range beside a timestamp on a public page is often enough to name an employer, which is a thing a visitor reading a portfolio at nine at night did not sign up for. Truncating the address does not fix that; it only shortens the sentence. **The email rule is absolute.** No email address appears anywhere in this feature: not in a response, not on the page, not in a tooltip, not in a log line. A rule like that is not kept by remembering it. It is kept by a shape that has no room for the thing. ## Decision **The row is written by the store that served the request.** Two adapters behind one port, the same shape as bids: two counter tables on the relational side, two kinds of counter document in one container on the document side. The store split the ask wants comes free, because the row lives where the request was served, and the feature demonstrates the thing the site is about: the same shape on two engines, side by side, read back from both onto one graph. Application Insights was the other candidate: durable, already paid for, no new write path. It lost on the split and on the tests. The store does not appear in its request telemetry, adding it means a telemetry initializer and a query per Admin load through a managed identity, and none of it runs on a developer's machine or in the gate, where the card would only ever show its "not configured" state. A feature the gate cannot exercise is a feature the gate cannot hold. **Nothing waits on the store.** A request offers its hit to a bounded channel and leaves; a hosted service drains the channel every five seconds, or sooner when five hundred are waiting, folds the batch into one delta per store and hour and one per store, day and visitor, and hands each store its own. A full channel drops the oldest hit rather than blocking a request, and says so on the card. The Performance page's numbers are the reason: a synchronous write on every request would tax exactly the milliseconds that page is about. **The counters add; they do not overwrite.** Two containers write the same Azure SQL rows and the same Cosmos DB documents, the SQL site's and the document site's. On SQL Server the counts move by `ExecuteUpdate`, so the increment happens in the database; on Cosmos DB by a partial update whose `Increment` the service applies. The one field that is read, merged and written back is the JSON object of the top twenty paths, which a batch can lose to a batch from the other container in the same five seconds. A count of which paths were popular is worth exactly that much, and the trade is recorded here rather than hidden. **The activity writes are not on the Admin tab's other cards.** The relational adapter uses a context factory without the SQL log interceptor and the document adapter calls the container directly rather than through the measured helpers, on purpose: a batch every five seconds would fill the two-hundred-slot SQL log and store log with the feature that reads them, the observer effect the request ring already had to design out. What the feature costs is counted by the adapters instead and reported with the card. ### What a hit carries, and what it cannot ```csharp // Site activity, kept by the store that served the request (ADR: Site // activity, and the line an address does not cross). One port, two adapters, // the same shape as bids: the relational side keeps two tables, the document // side keeps two kinds of document in one container, and the Admin tab reads // either through this. // // What a hit carries is the whole privacy decision, so it is decided here and // not in an adapter. Seven fields: when, a visitor token that is a keyed hash // of the address and rotates daily, the address cut to its first three octets, // the path with no query string, the store, whether the request looked like a // bot, and on a page load the host of the page that linked here (the // host only, never its path or query, and not personal information, so it is // as public as the rest of the tab). No user agent, no // account, no email, no full address, no query string. A row that has no // field for a thing cannot leak it. /// One request as the activity feature sees it. The type has no room for anything a person could be named by. /// When the request arrived. /// A keyed hash of the address that rotates daily, so one visitor can be counted without being named. /// The address cut to its first three octets. /// The path requested, with no query string. /// The store that served the request. /// True when the request looked like a bot. /// On a page load, the host of the page that linked here; null otherwise. public sealed record ActivityHit(DateTimeOffset At, string Visitor, string Network, string Path, string Store, bool Bot, string? Source = null); /// One store's requests in one UTC hour, with the paths it served most, for the graph. /// The store that served the requests. /// The start of the UTC hour. /// How many requests the store served in the hour. /// How many of those requests looked like bots. /// The most requested paths in the hour, with a request count for each. public sealed record ActivityHour(string Store, DateTimeOffset Hour, int Requests, int Bots, IReadOnlyDictionary Paths); /// One visitor token on one store on one UTC day, for the table nobody anonymous can read. /// The store the visitor reached. /// The UTC day as yyyy-MM-dd. /// The visitor token, a keyed hash of the address that rotates daily. /// The address cut to its first three octets. /// The visitor's first request that day. /// The visitor's last request that day. /// How many requests the visitor made that day. /// How many of those requests looked like bots. /// The visitor's most requested paths, with a request count for each. /// The hosts that linked the visitor here, with a page-load count for each; null when none were kept. public sealed record ActivityVisitor( string Store, string Day, string Visitor, string Network, DateTimeOffset FirstSeen, DateTimeOffset LastSeen, int Requests, int Bots, IReadOnlyDictionary Paths, IReadOnlyDictionary? Sources = null); /// /// Whether a store can keep activity right now, and if not, why, in one /// sentence a page can show. is how long a kept /// row lasts, said by the store that keeps it ("kept in Azure Cosmos DB with no /// expiry"); null when the store is down or does not know. /// /// True when the store can keep activity right now. /// One sentence a page can show saying why activity is or is not kept. /// How long a kept row lasts, as the store says it; null when the store is down or does not know. public sealed record ActivityAvailability(bool Available, string Reason, string? Retention = null); /// What keeping activity has cost a store: request units, operations, and the operations that failed. /// The request units spent keeping activity. /// How many store operations keeping activity took. /// How many of those operations failed. public sealed record ActivityCost(double RequestUnits, int Operations, int Failures); /// /// What keeping activity has cost a store since the process started, for a /// store that has a unit to count it in: request units on the document store. /// Beside the port rather than on it, because the relational store has no such /// unit and a fake in a test should not have to invent one. /// public interface IActivityCost { ActivityCost Cost { get; } } /// Port: where activity is kept. Writes arrive in batches, never from a request thread. public interface IActivityStore { Task AvailabilityAsync(CancellationToken cancellation); Task RecordAsync(IReadOnlyList hits, CancellationToken cancellation); Task> HoursAsync(DateTimeOffset since, CancellationToken cancellation); Task> VisitorsAsync(DateTimeOffset since, CancellationToken cancellation); } /// The port wired to nothing: a store that did not come up keeps no activity and says so. public sealed class NullActivityStore(string reason) : IActivityStore { public static readonly NullActivityStore Instance = new("this store keeps no activity"); public Task AvailabilityAsync(CancellationToken cancellation) => Task.FromResult(new ActivityAvailability(false, reason)); public Task RecordAsync(IReadOnlyList hits, CancellationToken cancellation) => Task.CompletedTask; public Task> HoursAsync(DateTimeOffset since, CancellationToken cancellation) => Task.FromResult>([]); public Task> VisitorsAsync(DateTimeOffset since, CancellationToken cancellation) => Task.FromResult>([]); } ``` *Live from [`api/TheYard.Application/Activity.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Application/Activity.cs#L4-L116), region `activity-port`, read from this build at 1d6b50b.* Six fields. When; a visitor token; the network; the path; the store; whether it looked like a bot. No user agent, no account, no email, no query string, no full address. A type with no field for a thing cannot carry it, and the tests assert the consequence on the wire rather than in prose: after requests that put an address in the path and in the query string, neither response contains an at sign. **The visitor token** is a keyed hash of the day and the address, keyed with the signing key every container shares and nobody outside has. Within a day the same address is the same token on both sites, so a visitor's requests group; tomorrow it is a different token, so nothing joins across days; and the key is what stops anybody turning a token back into an address by hashing the whole IPv4 space, which without a key takes an afternoon. **The network** is the address cut to its first three octets, `203.0.113.x`, which is enough to see a network, a country and an obvious scanner range and not enough to name a machine. **A full address is never stored, on either store.** That is the default and it stands until the owner rules otherwise in writing; the record will say so here if he does. ```csharp /// /// The address, made into something the page can group by and nobody can /// reverse. A keyed hash of the day and the address: keyed with the signing /// key, which every container shares and nobody outside has, so the same /// address is the same token on both sites within a day and a different one /// tomorrow; the day is in the hash rather than in a salt table, so there is /// no table of salts to keep or to lose. The address itself is kept only as /// its first three octets, which is enough to see a network, a country and an /// obvious scanner range, and not enough to name a machine. A full address is /// never stored, and a full address is never sent anywhere from here. /// public sealed class VisitorTokens(string key) { /// The signing key as bytes, the key of every hash this makes. private readonly byte[] _key = Encoding.UTF8.GetBytes(key); /// Thirty-two hex characters: the first sixteen bytes of the keyed hash. public string TokenFor(string address, DateTimeOffset at) { byte[] hash = HMACSHA256.HashData(_key, Encoding.UTF8.GetBytes(at.UtcDay + "|" + address)); return Convert.ToHexStringLower(hash.AsSpan(0, 16)); } /// 203.0.113.7 becomes 203.0.113.x; an IPv6 address keeps its first three groups and ends in x; anything unreadable is x. public static string NetworkOf(string? address) { if (string.IsNullOrWhiteSpace(address)) { return "x"; } string[] octets = address.Split('.'); if (octets.Length == 4 && octets.All(octet => int.TryParse(octet, out int value) && value is >= 0 and <= 255)) { return $"{octets[0]}.{octets[1]}.{octets[2]}.x"; } if (address.Contains(':', StringComparison.Ordinal)) { string[] groups = address.Split(':'); return string.Join(':', groups.Take(3)) + ":x"; } return "x"; } /// /// The visitor's address. The forwarded-headers middleware has already written /// it onto the connection by the time anything reads it, counted back from the /// right of X-Forwarded-For past the proxies this app runs behind, so for a visitor /// who comes through the edge, a value they typed into the header is never the one /// used (ADR: The order of the request pipeline). Until then this read the edge's /// own header, and without it the leftmost X-Forwarded-For entry, which is the one /// a visitor writes. An address is what feeds a count, never a permission. /// public static string AddressOf(HttpContext context) => WithoutPort(context.Connection.RemoteIpAddress?.ToString() ?? ""); /// /// The address without the port a forwarding hop may have written after /// it. Since the forwarded-headers middleware resolves the address, the /// connection's address never carries a port and this is a guard rather than /// a fix; it stays because the rule it holds still matters. App Service's own /// requests reached the container as 127.0.0.1 with a port, and with the port /// left on, it went into the token and the /// network: every one of those requests was a new visitor, hundreds a day, /// and the network showed the whole address. 203.0.113.7:443 is /// 203.0.113.7; [2001:db8::1]:443 is 2001:db8::1; an address with no port /// is itself. /// public static string WithoutPort(string address) { if (address.StartsWith('[')) { int close = address.IndexOf(']', StringComparison.Ordinal); return close > 1 ? address[1..close] : address; } int colon = address.IndexOf(':', StringComparison.Ordinal); return colon > 0 && colon == address.LastIndexOf(':') && address[..colon].Count(c => c == '.') == 3 ? address[..colon] : address; } } ``` *Live from [`api/TheYard.Api/Activity.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Activity.cs#L25-L108), region `visitor-token`, read from this build at 1d6b50b.* **A path with an at sign in it** is recorded with the sign percent-encoded, so an address pasted into a URL, which scanners do, cannot travel. The query string is never recorded at all. ### Two endpoints, not one **`GET /api/admin/activity`** is public, like the rest of the tab. It carries the series per store on a fixed grid of buckets with zeros where nothing happened, so the two lines share an axis; the totals; the split between what looked like people and what looked like scanners; the top paths; which stores keep activity here and why not; and what the collector has done. It names nobody, and a test holds it to that. **`GET /api/admin/activity/visitors`** is the one Admin read that is not public. It answers only to the operator's key, presented as a header or a query parameter and compared in constant time, and a wrong key, a missing key and an unconfigured key are all a 404, so a stranger cannot tell the endpoint exists. The key is `Admin__Key` in the container, filled at roll time from a repository secret like the signing key; until the secret exists, the endpoint does not. The page fetches the rows only when the address bar carries the key and sends it in a header, so without the key the table does not exist in the page any more than it exists on the wire. The decision on that split was put to Steve with the two options priced: rows behind a key only he holds, or aggregate only and no rows. He was away; the key-protected shape is the default because it exposes nothing and keeps the view he asked for, and he can rule the other way by deleting one endpoint. ```csharp /// /// The report both endpoints are built from. The public one carries the /// series, the totals and the top paths and names nobody; the keyed one /// carries the visitor rows. Both are read from every store this container /// runs, so the graph shows the two stores against each other from one /// container's point of view, which is the point of writing the row where /// the request was served. /// public static class ActivityReport { /// /// The public report for a window: the series per store on a shared grid, /// the totals, unique visitors per day, who the traffic was, the top /// paths, which store keeps the rows and how long, what the collector has /// done, and what the feature has cost the keeper. It names nobody: the /// visitor rows are read only to be counted. /// public static async Task PublicAsync(ActivityCollector collector, Backends backends, string window, DateTimeOffset now, bool visitorRows, CancellationToken cancellation) { var chosen = ActivityWindows.Parse(window)!.Value; DateTimeOffset since = (now - chosen.Length).UtcHour; var stores = new List(); var series = new List(); var paths = new Dictionary(StringComparer.Ordinal); int requests = 0; int bots = 0; var byStore = new List(); var keptRows = new List(); // One read, from the keeper: every store's rows live there, each one // naming the store that served it, so the two lines still show against // each other and a paused relational database cannot take the card // down with it. var keeper = collector.Keeper; var availability = await keeper.AvailabilityAsync(cancellation); IReadOnlyList kept = availability.Available ? await keeper.HoursAsync(since, cancellation) : []; // The visitor rows are read here for counts and thrown away: how many // distinct tokens each day saw, of which kind, and what each kind // asked for. The rows themselves leave the server only through the // keyed endpoint below. IReadOnlyList keptVisitors = availability.Available ? await keeper.VisitorsAsync(since, cancellation) : []; foreach (var backend in backends.All) { stores.Add(new { store = backend.Key, name = backend.Name, available = availability.Available, reason = availability.Reason, kept_by = collector.KeeperKey }); var hours = kept.Where(hour => hour.Store == backend.Key).ToList(); var points = Bucket(hours, since, now, chosen.Bucket); series.Add(new { store = backend.Key, name = backend.Name, points }); keptRows.AddRange(keptVisitors.Where(visitor => visitor.Store == backend.Key && visitor.LastSeen >= since)); int storeRequests = hours.Sum(hour => hour.Requests); int storeBots = hours.Sum(hour => hour.Bots); requests += storeRequests; bots += storeBots; byStore.Add(new { store = backend.Key, requests = storeRequests, bots = storeBots, humans = storeRequests - storeBots }); foreach (var hour in hours) { foreach (var (path, count) in hour.Paths) { paths[path] = paths.GetValueOrDefault(path) + count; } } } var counters = collector.Counters; // What the feature has cost the keeper since this process started, when // the keeper counts it (the document store does, in request units); // null on a store that has no such unit. var cost = (collector.Keeper as IActivityCost)?.Cost; return new { window = chosen.Name, // Whether the per-visitor rows are served on this site at all, so // the page knows whether to offer them (off by default). visitor_rows = visitorRows, bucket = chosen.Bucket == TimeSpan.FromDays(1) ? "day" : chosen.Bucket == TimeSpan.FromHours(6) ? "six hours" : "hour", since, until = now, totals = new { requests, bots, humans = requests - bots }, by_store = byStore, series, days = Days(keptRows, backends.All.Select(backend => backend.Key).ToList(), since, now), // Who the traffic was, from the same visitor rows the days are: // people, scanners and crawlers, and the site reading itself, so // the card can show visitors only or everything. who = ActivityWho.Summary(keptRows, backends.All.Select(backend => backend.Key).ToList()), // The resume's opens by the door its link named, people only (ADR: The resume has an address). doors = ActivityWho.Doors(keptRows), top_paths = paths.OrderByDescending(entry => entry.Value).ThenBy(entry => entry.Key, StringComparer.Ordinal).Take(12) .Select(entry => new { path = entry.Key, requests = entry.Value }).ToList(), stores, kept_by = collector.KeeperKey, // The keeper's own sentence for how long the rows are kept ("kept in // Azure Cosmos DB with no expiry"), its own field on the port rather // than the reason a store is down. retention = availability.Available ? availability.Retention : null, collector = new { offered = counters.Offered, written = counters.Written, failed_batches = counters.FailedBatches, dropped = counters.Dropped, last_write = counters.LastWrite, interval_seconds = (int)ActivityCollector.Interval.TotalSeconds, }, cost = cost is null ? null : new { request_units = Math.Round(cost.RequestUnits, 2), operations = cost.Operations, failures = cost.Failures }, }; } /// /// The keyed report for a window: one row per visitor token and day, with /// its network, store, first and last request, counts and top five paths, /// for the stores this container runs. Served only behind the operator's /// key (AdminKey), and capped at five hundred rows. /// public static async Task VisitorsAsync(ActivityCollector collector, Backends backends, string window, DateTimeOffset now, CancellationToken cancellation) { var chosen = ActivityWindows.Parse(window)!.Value; DateTimeOffset since = now - chosen.Length; var rows = new List(); var known = backends.All.Select(backend => backend.Key).ToHashSet(StringComparer.Ordinal); foreach (var visitor in await collector.Keeper.VisitorsAsync(since, cancellation)) { if (visitor.LastSeen < since || !known.Contains(visitor.Store)) { continue; } rows.Add(new { visitor = visitor.Visitor, network = visitor.Network, store = visitor.Store, day = visitor.Day, first_seen = visitor.FirstSeen, last_seen = visitor.LastSeen, requests = visitor.Requests, bots = visitor.Bots, top_paths = visitor.Paths.OrderByDescending(entry => entry.Value).ThenBy(entry => entry.Key, StringComparer.Ordinal).Take(5) .Select(entry => new { path = entry.Key, requests = entry.Value }).ToList(), }); } return new { window = chosen.Name, since, until = now, count = rows.Count, // Newest first, and no more than five hundred: a table longer than // that is a file, not a page. visitors = rows.Take(500).ToList(), }; } /// /// Unique visitors per UTC day, the graph's series: every unique address /// per day, as the owner asked for it. One row per day in the window, /// zeros where nobody came: the distinct tokens that day across every /// store, the distinct tokens per store, and how many of them looked like /// people. A token is one address for one day, so "unique visitors" here /// is unique addresses, counted without keeping one. Each day also says /// who: people, scanners and crawlers, and the site's own reads, the three /// adding up to the day's visitor-days (ActivityWho has the rule). /// private static List Days(List rows, IReadOnlyList stores, DateTimeOffset since, DateTimeOffset now) { var days = new List(); var first = new DateTimeOffset(since.Year, since.Month, since.Day, 0, 0, 0, TimeSpan.Zero); var last = new DateTimeOffset(now.Year, now.Month, now.Day, 0, 0, 0, TimeSpan.Zero); for (var at = first; at <= last; at = at.AddDays(1)) { string day = at.UtcDay; var today = rows.Where(row => row.Day == day).ToList(); var tokens = today.Select(row => row.Visitor).Distinct(StringComparer.Ordinal).ToList(); // A visitor is a person if any store saw a request of theirs that did not look like a bot. int humans = tokens.Count(token => today.Any(row => row.Visitor == token && row.Bots < row.Requests)); var who = ActivityWho.Count(today); days.Add(new { day, visitors = tokens.Count, humans, bots = tokens.Count - humans, people = who.People, scanners = who.Scanners, self = who.Self, by_store = stores.Select(store => { var theirs = today.Where(row => row.Store == store).ToList(); var split = ActivityWho.Count(theirs); return new { store, visitors = theirs.Select(row => row.Visitor).Distinct(StringComparer.Ordinal).Count(), people = split.People, scanners = split.Scanners, self = split.Self, }; }).ToList(), }); } return days; } /// The hours of one store laid onto a fixed grid of buckets from since to now, zeros where nothing happened, so the two lines share an x axis. private static List Bucket(IEnumerable hours, DateTimeOffset since, DateTimeOffset now, TimeSpan bucket) { var start = bucket == TimeSpan.FromDays(1) ? new DateTimeOffset(since.Year, since.Month, since.Day, 0, 0, 0, TimeSpan.Zero) : new DateTimeOffset(since.Year, since.Month, since.Day, since.Hour - (since.Hour % (int)Math.Max(1, bucket.TotalHours)), 0, 0, TimeSpan.Zero); var counts = new SortedDictionary(); for (var at = start; at <= now; at += bucket) { counts[at] = (0, 0); } foreach (var hour in hours) { long index = (long)Math.Floor((hour.Hour - start) / bucket); if (index < 0) { continue; } var at = start + (bucket * index); var (requests, bots) = counts.GetValueOrDefault(at); counts[at] = (requests + hour.Requests, bots + hour.Bots); } return counts.Select(entry => (object)new { at = entry.Key, requests = entry.Value.Requests, bots = entry.Value.Bots }).ToList(); } } ``` *Live from [`api/TheYard.Api/ActivityReport.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/ActivityReport.cs#L10-L245), region `report`, read from this build at 1d6b50b.* ### The graph An inline SVG in the palette, the same two colours the drawing on the SQL Server and Cosmos DB page uses for its numbers: the accent for the relational store, the taupe deepened for the document store, both held to 3:1 on white and on the page ground by the palette test, because a line is a graphic and that is the floor for one. No chart library: the drawings under `docs/images` are hand-drawn SVG and this is drawn the same way, with the geometry in a module a unit test can hold. Three windows, the last day by the hour, the last week by six hours, the last month by the day. ```ts /** * The drawing area the lines are laid into, in SVG units: the desk's, which the * card scales up to its width, and narrowed to the width a phone gives it * (fitBox in src/lib/plotFrame.ts) so its words keep their size. */ export const CHART = { width: 720, height: 200, left: 36, right: 12, top: 12, bottom: 28 } as const; /** * The axis's top for a peak: the next round number (1, 1.2, 1.5, 2, 2.5, 3, 4, 5, 6, 8 of a power of * ten, whole) with a tenth of headroom over it, so the highest day never touches the frame and * the label stays a number a reader expects. 1 for an empty window, so a flat day still has an axis. */ export function headroom(peak: number): number { if (peak <= 0) return 1; const want = peak * 1.1; const scale = 10 ** Math.floor(Math.log10(want)); for (const step of [1, 1.2, 1.5, 2, 2.5, 3, 4, 5, 6, 8, 10]) { const top = step * scale; if (top >= want && Number.isInteger(top)) return top; } return 10 * scale; } /** The axis's top over the highest count on any series: headroom() of the peak. */ export function ceilingOf(series: ActivitySeries[]): number { let top = 0; for (const line of series) { for (const point of line.points) { if (point.requests > top) top = point.requests; } } return headroom(top); } /** * One series as the `d` of an SVG path: a polyline through every point, left * to right, scaled to the drawing area. Points sit at equal x steps because * the server hands back a fixed grid with zeros where nothing happened, so * the two stores share an axis without the page having to align them. */ export function linePath(points: ActivityPoint[], ceiling: number, box: PlotBox = CHART): string { if (points.length === 0) return ''; const innerWidth = box.width - box.left - box.right; const innerHeight = box.height - box.top - box.bottom; const step = points.length === 1 ? 0 : innerWidth / (points.length - 1); return points .map((point, index) => { const x = box.left + index * step; const y = box.top + innerHeight - (point.requests / ceiling) * innerHeight; return `${index === 0 ? 'M' : 'L'}${x.toFixed(1)} ${y.toFixed(1)}`; }) .join(' '); } /** The same line closed down to the baseline, for the soft fill under it. */ export function areaPath(points: ActivityPoint[], ceiling: number, box: PlotBox = CHART): string { const line = linePath(points, ceiling, box); if (line === '') return ''; const innerWidth = box.width - box.left - box.right; const baseline = box.height - box.bottom; const step = points.length === 1 ? 0 : innerWidth / (points.length - 1); const lastX = box.left + (points.length - 1) * step; return `${line} L${lastX.toFixed(1)} ${baseline} L${box.left} ${baseline} Z`; } // #region stack /** * The three kinds of traffic stacked: people at the bottom, then scanners and * crawlers, then the site's own reads, in that order always, so a colour means * the same kind on every window. Under Visitors only the stack is the people * alone. A band is the day-by-day floor and ceiling it fills between, in * visitor-days. */ export type ActivityKind = 'people' | 'scanners' | 'self'; /** The three kinds in stacking order, bottom first. */ export const ACTIVITY_KINDS: readonly ActivityKind[] = ['people', 'scanners', 'self']; /** Each kind as the legend and the table name it. */ export const KIND_NAMES: Readonly> = { people: 'People', scanners: 'Scanners and crawlers', self: "The site's own reads", }; /** One kind's band: its floor and its ceiling on each day, in visitor-days. */ export type ActivityBand = { kind: ActivityKind; lower: number[]; upper: number[] }; /** The bands for the toggle, each one starting where the one below it ends, day by day. */ export function stackBands(days: ActivityDay[], who: ActivityWho): ActivityBand[] { const kinds: readonly ActivityKind[] = who === 'people' ? ['people'] : ACTIVITY_KINDS; const running = days.map(() => 0); return kinds.map((kind) => { const lower = [...running]; days.forEach((day, index) => { running[index] += day[kind]; }); return { kind, lower, upper: [...running] }; }); } /** The axis's top over the stack on its highest day: headroom() of it, 1 for a quiet window. */ export function stackCeiling(bands: ActivityBand[]): number { const top = bands.length === 0 ? [] : bands[bands.length - 1].upper; return headroom(Math.max(0, ...top)); } /** The x of the day at `index` of `count`, spread evenly across the drawing area. */ export function xAt(index: number, count: number, box: PlotBox = CHART): number { const innerWidth = box.width - box.left - box.right; return box.left + (count <= 1 ? 0 : (innerWidth / (count - 1)) * index); } /** The y of a count against the ceiling, zero at the baseline; the chart's height never narrows, so it reads CHART. */ export function yAt(value: number, ceiling: number): number { const innerHeight = CHART.height - CHART.top - CHART.bottom; return CHART.top + innerHeight - (value / ceiling) * innerHeight; } /** One band as the `d` of a closed SVG path: along its ceiling left to right, back along its floor. */ export function bandPath(band: ActivityBand, ceiling: number, box: PlotBox = CHART): string { const count = band.upper.length; if (count === 0) return ''; const top = band.upper.map( (value, index) => `${index === 0 ? 'M' : 'L'}${xAt(index, count, box).toFixed(1)} ${yAt(value, ceiling).toFixed(1)}` ); const bottom = band.lower .map( (value, index) => `L${xAt(index, count, box).toFixed(1)} ${yAt(value, ceiling).toFixed(1)}` ) .reverse(); return `${[...top, ...bottom].join(' ')} Z`; } /** A band's ceiling alone, drawn in the card's ground over the fills: the two-pixel gap between bands. */ export function edgePath(band: ActivityBand, ceiling: number, box: PlotBox = CHART): string { const count = band.upper.length; return band.upper .map( (value, index) => `${index === 0 ? 'M' : 'L'}${xAt(index, count, box).toFixed(1)} ${yAt(value, ceiling).toFixed(1)}` ) .join(' '); } /** * Where a band's name is written on the chart: the middle of the day it is * thickest, anchored so it never leaves the drawing at either end; nowhere * when the band is never as thick as a line of text, in which case the legend * and the table carry it. */ export function labelSpot( band: ActivityBand, ceiling: number, box: PlotBox = CHART, minHeight = 16 ): { x: number; y: number; anchor: 'start' | 'middle' | 'end' } | null { const count = band.upper.length; let best = -1; let thickest = 0; band.upper.forEach((value, index) => { const thickness = yAt(band.lower[index], ceiling) - yAt(value, ceiling); if (thickness > thickest) { thickest = thickness; best = index; } }); if (best < 0 || thickest < minHeight) return null; return { x: xAt(best, count, box) + (best === 0 ? 6 : best === count - 1 ? -6 : 0), y: (yAt(band.upper[best], ceiling) + yAt(band.lower[best], ceiling)) / 2 + 4, anchor: best === 0 ? 'start' : best === count - 1 ? 'end' : 'middle', }; } /** * Today's column, when the window ends on today (UTC): which point it is and * how many of its hours have passed, so the chart can say the last day is a * part-day and not a fall. */ export function partialDay( days: { day: string }[], now: Date ): { index: number; hours: number } | null { const index = days.length - 1; return index >= 0 && days[index].day === now.toISOString().slice(0, 10) ? { index, hours: now.getUTCHours() } : null; } /** * What a part-day says of itself: above the drawing at its right end, and in * the crosshair's box, never on the axis, where at a week it ran into the day * before it. The day is a UTC day and the hours are UTC hours, so the words * say UTC: at 8 am in Central time the day is already 13 hours in. */ export function todayNote(hours: number): string { return `today (UTC), ${hours} h in`; } /** The day nearest a point along the drawing, in the drawing's own units, never off either end. */ export function dayAt(x: number, count: number, box: PlotBox = CHART): number { if (count <= 1) return 0; const innerWidth = box.width - box.left - box.right; const index = Math.round(((x - box.left) / innerWidth) * (count - 1)); return Math.min(count - 1, Math.max(0, index)); } // #endregion stack /** Which points carry an x label: about six across the width, the first and the last always. */ export function labelledIndexes(count: number): number[] { if (count <= 1) return count === 1 ? [0] : []; const every = Math.max(1, Math.round((count - 1) / 5)); const indexes: number[] = []; for (let index = 0; index < count; index += every) indexes.push(index); if (indexes[indexes.length - 1] !== count - 1) indexes.push(count - 1); return indexes; } /** A day's x label: the weekday for a week, the date for a month, and the date for a day too, since a day window is two days. */ export function labelFor(at: string, window: ActivityWindow): string { // A UTC day, `2026-09-13`, read as that calendar day and not as the // instant before it in a western zone. const date = new Date(`${at}T12:00:00Z`); if (window === '7d') { return date.toLocaleDateString(undefined, { weekday: 'short', timeZone: 'UTC' }); } return date.toLocaleDateString(undefined, { month: 'short', day: 'numeric', timeZone: 'UTC' }); } ``` *Live from [`src/lib/activityChart.ts`](https://github.com/SteveStout/TheYard/blob/1d6b50b/src/lib/activityChart.ts#L16-L244), region `chart-geometry`, read from this build at 1d6b50b.* ### What the feature costs the stores Measured on the gate's Cosmos DB pass: the collector's own counters and the document adapter's request-unit total are on the card, so the number a reader sees is the number the container paid, not a number written here that goes stale. A hit is one partial update per hour and one per visitor per batch, on the free tier's shared thousand request units a second, and the container's documents expire after thirty-five days so it cleans itself. The effect on the proof card, the paired rounds that the Performance page quotes, is the measurement this record owes: the run before this change on the second container read 41 ms and 2 ms round trips, and the run after it ships is recorded in the addendum below when it has been read off the live site, not before. ## Alternatives **Application Insights, queried per Admin load.** Above: no store split without a change to what is sent, and nothing to hold in the gate. **Full addresses, behind the key.** Rejected by default and reversible in writing. The data is written before it is read, so a decision to store the address is a decision about every row from then on, and the operator who wants it should say so where the record can quote him. **The rows on the public endpoint, truncated.** Rejected. A `/24` and a timestamp on a public page can name an employer, and the tab is public by design (ADR: The one write a stranger can make). **Storing the user agent.** It would sharpen the bot guess. It would also be a fingerprint beside a network and a time, on rows an operator's key opens, and a guess that is right often enough does not need it. ## Consequences - The Admin tab opens on the graph and the two stores show against each other over a day, a week or a month, from durable rows rather than a ring that dies with the container. - Two tables to publish to Azure SQL and one container to apply on Cosmos DB, by a person, like every other schema change here (ADR: Data first, and the database in source control). Until they are, each store's card line says what is missing and keeps nothing. - One secret to add for the visitor rows to exist on the live sites. - The test count moves, and so does the number on the resume that quotes it. ## Where it sits Site activity crosses the back end from Application outward, and the front end: ActivityHit and the IActivityStore port sit in Application, EfActivityStore in Infrastructure and CosmosActivityStore in Infrastructure.Cosmos implement that port, the collector and visitor token live in the host Api, and the chart geometry is React-free code in src/lib. Application owns the port and both adapters fill it in (dependency inversion), which is why one keeper can take every row without the collector changing. It cost a second schema to publish by hand and a top-paths field that can lose a batch when both containers write in the same five seconds. Application Insights would be the better home if the serving store could travel as a custom dimension and the gate had no need to exercise the card. ## Files - [`api/TheYard.Application/Activity.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Application/Activity.cs): the port, the hit, and the folding of a batch into deltas. - [`api/TheYard.Api/Activity.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Activity.cs): the token, the bot guess, the collector and the report. - [`api/TheYard.Api/AdminKey.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/AdminKey.cs): the key, read from the query or the header in one place and compared in constant time. - [`api/TheYard.Api/Composition/ObservabilityRegistration.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Composition/ObservabilityRegistration.cs): the hook beside the request ring (region activity-hook). - [`api/TheYard.Api/Composition/ActivityRegistration.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Composition/ActivityRegistration.cs): the wiring. - [`api/TheYard.Api/Endpoints/AdminEndpoints.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Api/Endpoints/AdminEndpoints.cs): the two endpoints (region activity-endpoints). - [`api/TheYard.Infrastructure/EfActivityStore.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Infrastructure/EfActivityStore.cs) and [`ActivityRows.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Infrastructure/ActivityRows.cs): the relational adapter and its two rows. - [`api/TheYard.Database/Tables/ActivityHours.sql`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Database/Tables/ActivityHours.sql) and [`ActivityVisitors.sql`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Database/Tables/ActivityVisitors.sql): the schema, which is the authority. - [`api/TheYard.Infrastructure.Cosmos/CosmosActivityStore.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Infrastructure.Cosmos/CosmosActivityStore.cs) and [`infra/cosmos/activity.json`](https://github.com/SteveStout/TheYard/blob/main/infra/cosmos/activity.json): the document adapter and the container it needs. - [`src/lib/activity.ts`](https://github.com/SteveStout/TheYard/blob/main/src/lib/activity.ts) and [`src/components/admin/ActivityCard/ActivityCard.tsx`](https://github.com/SteveStout/TheYard/blob/main/src/components/admin/ActivityCard/ActivityCard.tsx): the geometry and the card. - [`api/TheYard.Tests/ActivityTests.cs`](https://github.com/SteveStout/TheYard/blob/main/api/TheYard.Tests/ActivityTests.cs) and [`tests/e2e/admin.spec.ts`](https://github.com/SteveStout/TheYard/blob/main/tests/e2e/admin.spec.ts): the folding, the token, the endpoints, and the at sign that is never there. ```csharp // And the same request as a hit for the activity card, offered to the // collector and forgotten: nothing here waits on a store. What the hit // carries, and what it cannot, is decided in Activity.cs and in the type. // The token and the network are made once here and shared with the kept // log below, so a request pays for one keyed hash and not two. if (host.VisitorTokens is not { } visitorTokens || !Hits.Counts(path)) { return; } string address = VisitorTokens.AddressOf(context); var at = DateTimeOffset.UtcNow; string token = visitorTokens.TokenFor(address, at); string network = VisitorTokens.NetworkOf(address); string store = backends.For(context).Key; host.ActivityCollector?.Offer(Hits.For(visitorTokens, address, at, token, network, path, store, context.Request.Headers.UserAgent.FirstOrDefault(), Hits.SourceOf(context.Request.Headers.Referer.FirstOrDefault(), path, context.Request.Host.Host))); ``` *Live from [`api/TheYard.Api/Composition/ObservabilityRegistration.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Composition/ObservabilityRegistration.cs#L176-L191), region `activity-hook`, read from this build at 1d6b50b.* ```csharp // Site activity (ADR: Site activity, and the line an address does not cross). // The public one: requests over time split by store, the totals, the top // paths, the bot and human counts, and what the feature has cost. It names // nobody, so it is as public as the rest of this tab. The window is a name // and not a number, so a caller cannot ask for a year. // A report is kept thirty seconds and rebuilt behind the next read for ten // minutes after that, so a reader never waits on the visitor rows being counted // (ActivityReportCache; ADR: Site activity, and the line an address does not // cross, the addendum on the first read). app.MapGet("/api/admin/activity", Activity); // The visitor rows, behind the operator's key: a token that rotates daily, // the network to three octets, the store, the counts and the top paths. The // key is presented as a query parameter or a header and compared in constant // time; a wrong key, a missing key and an unconfigured key are all a 404, so // a stranger cannot tell the endpoint exists. This is the one Admin read that // is not public, because a network range beside a timestamp on a public page // can name an employer, and the tab is public by design (ADR-054). app.MapGet("/api/admin/activity/visitors", Visitors); ``` *Live from [`api/TheYard.Api/Endpoints/AdminEndpoints.cs`](https://github.com/SteveStout/TheYard/blob/1d6b50b/api/TheYard.Api/Endpoints/AdminEndpoints.cs#L62-L80), region `activity-endpoints`, read from this build at 1d6b50b.* ## Addendum, 2026-09-13: the measurement, read off both live containers on 1.0.0.114 The paired-round proof was run on both containers within a quarter of an hour of the roll, each started by a throwaway account made for it, with the activity collector writing to both stores the whole time. Read back from the cards: - **The live site's container:** 39 ms to Azure SQL Database and 2 ms to Azure Cosmos DB. "On 3 of 8 paths the two stores answer in the same time; 4 more differ by exactly the round trip to the store." Bid write 84 ms against 13, bid raise 87 against 11, sign in 128 against 83, the vehicle page 1 against 1. - **The second container:** 38 ms and 2 ms, the same sentence. Bid write 84 against 11, bid raise 83 against 11. Before the change, the second container's card from 1.0.0.112 read 41 ms and 2 ms with the same sentence, and the 1.0.0.94 run the Performance page quotes read 39 ms and 2 ms with bid write at 84 against 10 and raise at 85 against 9. Every row that differs does so by a few milliseconds in both directions, which is the noise the record on the proof already describes. The batch writer off the request path costs the request path nothing the card can see. What the feature itself cost during that read: the collector on the live site's container had offered 88 hits and written 77 with no failed batch a minute after the run; the second container 254 offered, 121 written, none failed. Both containers then read the same rows back, 127 on each store from either address, which is the row living where the request was served rather than in the container that happened to answer. Still owed and named here: the visitor rows answer 404 on both sites until the repository secret `ADMIN_KEY` exists. The graph does not need it. ## Addendum, 2026-09-13: unique visitors per day, and the table grouped by day On seeing the first version, Steve asked for the graph to count unique addresses per day, with a table below of each visitor's activity grouped by day, at the top of the tab, so he can see how people use the site. The graph now draws unique visitors per UTC day: everybody as one line in the heading colour, and one line per store under it in the store colours, on a window of a week by default. A "unique visitor" is a distinct visitor token for that day, which is a distinct address for that day, counted on the server from the visitor rows and thrown away: the public endpoint carries the count per day and per store and never a token. The requests-per-bucket series stays on the wire for the tests that hold it and for anyone who wants the old view. The table under the graph groups each visitor's day under a heading row for the day, newest day first, with the day's own count of visitors and requests in the heading; within a day the columns still sort either way. The rows are the same rows as before and travel the same way, behind the operator's key, which is the reading this addendum keeps until he says otherwise: the tab is public, and a network beside a timestamp is the thing the record above decided not to publish. The graph, the day counts and the split need no key. ```ts /** A day's visitor-days under the toggle: the people only, or the three kinds together. */ export function countFor(kinds: ActivityKinds, who: ActivityWho): number { return who === 'people' ? kinds.people : kinds.people + kinds.scanners + kinds.self; } /** * The graph's series: unique visitors per UTC day, one line for everybody and * one per store, on the point shape the line geometry already draws. Under * Visitors only every line counts people; under All traffic, all three kinds. */ export function dayLines( days: ActivityDay[], stores: string[], who: ActivityWho = 'all' ): ActivitySeries[] { const all: ActivitySeries = { store: 'all', name: who === 'people' ? 'People' : 'All traffic', points: days.map((day) => ({ at: day.day, requests: countFor(day, who), bots: day.scanners })), }; const perStore = stores.map((store) => ({ store, name: store, points: days.map((day) => { const entry = day.by_store.find((candidate) => candidate.store === store); return { at: day.day, requests: entry ? countFor(entry, who) : 0, bots: 0 }; }), })); return [all, ...perStore]; } /** The visitor rows grouped by UTC day, newest day first, with the day's own totals for its heading. */ export function groupByDay( rows: ActivityVisitor[] ): { day: string; visitors: number; requests: number; rows: ActivityVisitor[] }[] { const byDay = new Map(); for (const row of rows) { const list = byDay.get(row.day) ?? []; list.push(row); byDay.set(row.day, list); } return [...byDay.entries()] .sort(([a], [b]) => b.localeCompare(a)) .map(([day, dayRows]) => ({ day, visitors: new Set(dayRows.map((row) => row.visitor)).size, requests: dayRows.reduce((sum, row) => sum + row.requests, 0), rows: dayRows, })); } ``` *Live from [`src/lib/activityDays.ts`](https://github.com/SteveStout/TheYard/blob/1d6b50b/src/lib/activityDays.ts#L15-L64), region `days`, read from this build at 1d6b50b.* ## Addendum, 2026-09-13: the counters are kept for good The same afternoon, after the kept log shipped, Steve asked for long-term logs, of site activity above all. The thirty-five days above were chosen to match the graph's longest window and nothing else, and the documents are small: one per store per hour and one per visitor per store per day, a few hundred a day at today's traffic and a few tens of megabytes a year. So the `activity` container's default time-to-live is now none (`-1` in the definition, which keeps time-to-live enabled and expires nothing), applied to the live account by `az cosmosdb sql container update` and read back, and the relational side's two tables were never purged. The card says the retention it reads from the container rather than a sentence written here. The graph's windows stay at 24 hours, 7 days and 30 days: a longer window reads every visitor document in it to count the day's unique tokens, and a year of those is tens of thousands of reads per Admin load, which is a separate decision with a separate cost if he wants one. ## Addendum, 2026-09-13: the browser remembers the key An hour after the retention change, Steve reported from his phone that the kept log did not show on the public site, where he expected to see it. Measured before anything was changed: the runner opened the keyed URL in the repo's own headless Chromium and the card rendered with 130 lines, and the same page without the key showed the one-line note. The key was in a file on his machine, and he was reading the site from his phone. So the page now remembers the key. The address bar wins and is written to this browser's local storage; a visit without it reads what the browser kept; a button under the visitor table forgets it, at once and for the next visit. The key still never leaves the browser except as the header the two keyed endpoints read, the app still drops it from the address bar on the first render, and a bookmark saved after the page loaded now works, which it did not before. What this costs: anyone with that browser unlocked sees the operator's cards, which is the same trust the browser's saved sessions already carry, and the forget button is the answer on a shared machine. The browser suite opens the keyed URL once, then the plain one, and holds both the remembering and the forgetting. ## Addendum, 2026-09-13: the key can be typed in Steve, from his phone, with a screenshot: the keyed URL opened the Admin tab and the kept log card said the log answers only to the operator's key, which means the page loaded with no `key=` in its address. The same URL in an emulated iPhone from the runner rendered the card with two hundred rows, so the page and the key were right and the link's journey to the phone was not; how the query string was lost on the way is not measured and is not named. What is fixed is the dependence: the Kept log card, when it has no key, offers a box to type it into, remembers it on that browser exactly as the address bar would have, and opens both keyed cards at once. The box is a password field, the entry is trimmed, an empty entry does nothing, and the browser test types the key in after forgetting it and finds both cards open on the next plain visit. The keyed URL still works where it arrives whole; the operator no longer needs it to. ## Addendum, 2026-09-13: the rows are off Late that day, Steve asked for the per-visitor data to be switched off for now. Done as one setting rather than a removal, because the day's evidence is that this decision moves: `Admin:VisitorRows`, off by default. Off, the visitor table and the kept log answer 404 to everybody, with the key as without it, the public report says `visitor_rows: false`, and the page shows neither the table, the kept log card nor anything about them. The rows keep being written: the counters the graph is drawn from are the same documents, the kept log is the record he asked for an hour earlier, and turning the setting on shows what was kept meanwhile. The graph of unique visitors per day stays public; it names nobody. The operator's key stays for the one card that still needs it, the reset links, and its box moved to an Operator card of its own. A test holds the 404 with the key on a host with the rows at their default. ## Addendum, 2026-09-14: one keeper, and it is the document store The rule above, each store keeps the rows for the requests it served and the report reads both, met the free tier on the fourteenth. Azure SQL Database's free amount is 100,000 vCore-seconds a month, and a serverless database pauses only after an idle hour; a collector writing a batch every five seconds is never idle. The database that had paused between visits since the third of September stopped pausing on the thirteenth, when this record shipped, and at 06:40 UTC on the fourteenth Azure paused it for the rest of the month with its own sentence in the exception (error 42119): the free amount will renew on the first of October. The site kept serving, from its files, with no accounts and no bids, and the activity card on both sites answered 500, because the SQL half of the read threw before the Cosmos DB half was asked. Measured off `az sql db show` (status Paused, pausedDate 2026-09-14T06:40:21Z) and off the container's log, not inferred. Two changes, and Steve named the first: the activity should live on Cosmos DB, permanently. The collector now writes every batch to one keeper, Azure Cosmos DB wherever it is configured, whichever store served the request; the row still carries the serving store's key, so the graph keeps its line per store and the comparison stands. The report reads the keeper once and splits the rows by that key, and a store that is down cannot take the card with it. Without Cosmos DB on a container the default store keeps its own rows, which is what the test host does. The second is in the store's own record: the site now points at a Basic database beside the paused one (ADR: The SQL Server backend, addendum of 14 September). What it costs: the activity rows were already on Cosmos DB with no expiry (the addendum of the thirteenth); this moves the SQL site's share there too, a few hundred request units a day inside the free tier's thousand a second. What it stops costing is the whole of the free relational amount. A test holds the keeper: two stores, two hits naming each, both rows land on the keeper and none on the other. ## Addendum, 2026-09-24: people, scanners, and the site reading itself The card said 710 people in a day. Read off the kept rows for the eight days to 24 September before anything changed (activitylane, queue 1347): from 20 September, the day both sites moved to App Service, 375 to 643 of each day's "people" were tokens with one request each, all of them `/index.html`, all from the loopback address with a port after it (`127.0.0.1:8069`, one port per request), 643 of 757 on the twenty-third. Requests from the machine itself: the port went into the token, so every one was a new visitor, and into the network, which showed the whole address. What sent them is not attributed here, because a row keeps no user agent; the shape (loopback, one request, the landing page, a few hundred a day on each site) is what was measured. The session key compounded it until 24 September: with the placeholder key each process invented its own, so one address was a new token after every roll and on each site. Three kinds now, each visitor-day exactly one of them, adding up to the day's total, and a toggle on the card, Visitors only by default: - **The site's own reads.** A row from the loopback address, which is this machine by definition, read from the network the row already keeps, so the rows written before this addendum read the same way as the rows after; all of a day's loopback rows are one visitor-day, the machine. And a row kept under the self mark: the tools that read the site on its operator's behalf (the page sweep, the ship's readers, the card's own pictures) carry `TheYard-SelfRead` on their user agent, as do App Service's own agents (AlwaysOn, the health check), and such a request is kept under a token of its own (the keyed hash of the mark and the address) and the network `self:x`, so it never folds into the row of a person at the same address and names no network. A stranger can claim the mark and hide from the card; that costs a count of visitors and reaches nothing else. - **Scanners and crawlers.** A token whose every request looked like a bot, by its agent or by what it asked for, the rule this record started with. - **People.** Everybody else: a token any store saw make a request that did not look like a bot. A port after an address is dropped before the token and the network are made, so one machine is one visitor whatever port it came from. What a hit carries is unchanged: no user agent is kept, the agent is read on the request and forgotten, as it always was for the bot guess. The kinds and what each asked for are counted from the visitor rows the days already came from, one read, and returned beside the days (`who` in the report); the rows still leave the server only through the keyed endpoint. Tests hold the mark, the port, the loopback networks, the one-machine rule and the three kinds adding up. The chart, from 1.0.3.12, stacks the three kinds day by day rather than drawing one line per store: people at the bottom, then scanners and crawlers, then the site's own reads, in three tokens chosen as a set and checked for colour vision together (the Colour and style page, its chart colours), each band's name written on the band where it is as thick as a line of text, a legend for all three, and the day-by-day numbers in a table under the chart. The lines by store are the other view, a click away; the split by store stays in the line under the chart in both. The path to the resume, from 1.0.3.13: four steps, counted in visitor-days from the same rows, each a set of paths (the page, which every address the site serves is kept as; `/api/vehicles`, the inventory's listing; `/api/docs/author`; and the resume, `/api/docs/resume` as the page links it and `/docs/resume.pdf` as the repository serves it). Read off the rows before it shipped: of the 41 resume requests in the eight days to 24 September, 39 came through `/api/docs/resume`, which is why the step names it first. From 1.0.3.16 the card names what people asked for by page, from a table of patterns in `src/lib/activity.ts` (the most particular first, a path nothing names shown as itself), shows what scanners probed apart from it, and puts the collector's counts behind a Details line that says whether it is fine. ## Addendum, 2026-09-24: where they came from From 1.0.3.17 a hit carries a seventh field, on a page load only: the host of the page that linked here, lowercased, and nothing else of the referrer, so a path or a query that could carry something about the visitor never reaches a row. A referrer that is this site, the other store's site or the App Service origin is moving within the site and is no source; no referrer is kept as "(none)". The hosts are counted on the visitor row the way the paths are, top twenty, in both stores: a field on the Cosmos DB document, and a nullable `Sources` column on `ActivityVisitors` in the database project and in a SQLite migration. The relational column reaches Azure SQL Database at the next deliberate publish (ADR: Data first, and the database in source control). Until then the relational activity store finds the column missing and refuses itself, naming it, the way a missing table refuses it, rather than failing every batch; that matters only on a start where Cosmos DB did not answer, because Cosmos DB keeps the activity on both sites. The hosts are public on the card, beside the five groups the card draws (LinkedIn, GitHub, search, another site, typed or unknown). Three ways were put to Steve with a picture of each: the group alone, the plain host, and a keyed hash of the host. He chose the plain host, and on keeping it behind the operator's key: "no reason that should be behind a operator key no personal information", and "we would only obscure personal information and that is not personal". A referring site names a site and not a person. A link opened from a PDF, the resume among them, sends no referrer, so it reads as typed or unknown unless the link itself carries a tag. ## Addendum, 2026-09-25: what the card said it did, and what it did A read of this record against the code on 1.0.3.22 found the card promising three things it never showed and saying two things that had stopped being true. From 1.0.3.23: - **A dropped hit is counted.** The channel drops the oldest hit when it is full, and with that mode the write that causes the drop still succeeds, so the collector counted every offered hit as offered and nothing counted the drop, though this record and the collector's own summary said it did. The channel's callback for an evicted item now counts it; the report carries `collector.dropped` and the Details line says it. A test fills the channel past its ten thousand and reads the count. - **The request-unit cost is on the card.** The document adapter counted what the feature cost it from the start, as "What the feature costs the stores" above says the card shows, and nothing read it. The report carries `cost` (request units, operations, failures since the process started) when the keeper has a unit to count in, and null on the relational store, which has none; the Details line says it. - **The retention is on the card while the store is up.** The keeper's own sentence ("kept in Azure Cosmos DB with no expiry") reached the page only as the reason a store was unavailable. The report carries it as `retention` and the card shows it under the totals. - **One keeper, said so.** The Details line said "one batch per store" after the 14 September addendum made every batch go to one keeper; it names the keeper now, and so does the collector's summary. - **The About text says what the card shows.** It led with the per-store lines the card drew before 1.0.3.12 and described a visitor table behind a key, where the rows have been off since 13 September. It opens now with visitor-days stacked by kind and says no list of visitors is shown. On the 24 September addendum's sentence "What sent them is not attributed here": it was true of the rows, which keep no user agent, and the sender was named the same day off the configuration and the kept log (activitylane, queue 1353): both App Services run `alwaysOn` with `healthCheckPath` set to `/healthz`, and the kept log held 204 loopback lines per store that day, every one `GET /index.html`, a median of 300 seconds apart. The card has named App Service as the sender since 1.0.3.11; the sentence above stays as it was written. ## Addendum, 2026-09-25 (1.0.3.26): the retention and the cost, said precisely The card's retention line read the store's availability sentence, the reason a store gives when it is down, so on SQLite it said "kept in SQLite", a place and not a time. The port carries the retention as its own field from 1.0.3.26, and every store states it: Azure Cosmos DB from the container's time-to-live, the relational stores "with no expiry", which is what they do. The cost is a record, `ActivityCost`, counted with interlocked adds (the charge in hundredths of a request unit), because the batch writer and every report request count at once and a plain `+=` lost updates. The card says whose it is: this container's, since it started; the other container writes to the same keeper and counts its own. The About text no longer says no list of visitors is shown: the per-visitor rows, hashes only, are served to the operator's key on a site that turns them on. ## Addendum, 2026-09-28 (1.0.3.35): the first read, and where its two seconds went Measured on the Cosmos DB site from Missouri that morning, the first `GET /api/admin/activity` after a quiet spell took 2.0 to 2.2 s to its first byte, and the next about 260 ms. The workbench reads the week on mount and the card opens on the week, so that first read was the week's. Where it goes, timed step by step with a stopwatch against the live activity container from Steve's machine (a one-off test, read only, not kept; the figures include the trip from Missouri to the region, so the large reads are slower here than inside Azure, but the shape is the point): | Step | First use | Warm, two more passes | | --- | --- | --- | | The credential's first token | 2,964 ms | held | | The container's first read (the availability check) | 2,985 ms | 64 to 65 ms, then held | | The day's hour rows, 50 of them | 1,432 ms | 59 to 61 ms | | The day's visitor rows, 170 | 488 ms | 134 to 192 ms | | The week's visitor rows, 3,661 | 3,439 ms | 2,754 to 3,137 ms | | The month's visitor rows, 5,279 | 8,017 ms | 4,064 to 5,080 ms | | Folding the rows into the report | 46 to 73 ms | 1 to 101 ms | One read of each window cost 312 request units, nearly all of it the visitor rows. Two costs, then. A process's first use of the store pays for a token and for waking its connections, which nothing in the report can remove. And every report, first or not, reads every visitor row of its window to count the people, the scanners and the site's own reads: the largest cost of any read of the week or the month, and the one a reader was waiting on. **The fix: the report is kept, and rebuilt behind the reader** (`ActivityReportCache` in `api/TheYard.Api/Activity.cs`). A window's report younger than thirty seconds is served as it is. One older than that, up to ten minutes, is served at once while a new one is built behind it, so the next reader gets the new one. Only with no report, or one older than ten minutes, does a reader wait for the build, and however many ask at once, one build runs per window. The numbers on the card are at most thirty seconds behind the rows when anybody is reading it, and a reader after a quiet spell sees the last report while the new one is counted. The warm read is the time to serialise a report already built, under the 260 ms it was. Once the site is kept awake, a report is never ten minutes old, so no visitor waits on a build. `ActivityReportCacheTests` holds the thirty seconds, the ten minutes, the one build and a failed rebuild leaving the old report in place. **`/api/health`'s first read** (736 ms against about 190 warm) runs one probe per store (Azure SQL Database, and two point reads of Azure Cosmos DB) beside three file checks, each timed, and nothing it does is heavy. The difference is the connections: after a quiet spell both stores' connections have to be opened again, and the edge's to the origin with them. That is not a cost in this code, and the keep-warm loop is what answers it. ## Addendum, 2026-09-29 (1.0.3.48): a Site traffic section Steve proposed a section of the documentation on site traffic, like the one for styling, and then gave the go-ahead. The counting, the keeping and the finding were told only in two records and on the Admin tab's cards, and nothing tied them together for a reader who is not reading records. The Site traffic section, under How it is run beside Performance, is four pages on the Style section's pattern: a landing page with three tiles, the standing rules, a glossary and readouts counted from the build, then Who comes (what a visit carries, the three kinds, where readers came from, the path through), What is kept (the counters, the kept log, how long, public and keyed, what is never kept) and Being found (robots, the sitemap, /about, the structured data, llms.txt, Search Console). Every sentence on them is drawn from this record, ADR-073, ADR-053 and the code, and the pages show the code itself where it decides (the hit's fields and the visitor token, as live blocks). Nothing about how a visit is counted changed.