@karmaniverous/jeeves-scripts-core
    Preparing search index...

    lib/

    Shared infrastructure consumed by every domain: config-derived constants, CLI wrappers (gh, gog), the gateway client, worker dispatch and job-side Slack I/O, and the entity store. Configuration itself (schema, loader, getters, refs, silos, IMAP secrets) is in config.md. These modules are internal; the ones instance code may use are on the root export (constants, the date helpers, dispatchWithSlack, the component config readers, the gh helpers; see the package README).

    Fixed values plus the template's config-derived names, now read from the instance config (Decision 3). Nothing here is edited per instance: set the value in the config file.

    • Fixed exports (plain constants): TOKEN_METRICS_NAMESPACE, TOKEN_METRICS_CURSOR_KEY, TOKEN_METRICS_DB_CURSOR_KEY, TOKEN_METRICS_CC_CURSOR_KEY, SESSION_REFRESH_CONTEXT_THRESHOLD (150,000; renamed from SESSION_REFRESH_CACHE_READ_THRESHOLD, the trigger is the last turn's total context), SESSION_REFRESH_IDLE_MINUTES (60), OPENCLAW_UPGRADE_CUTOFF_ENV, and ENTITY_TYPES (subdir, rejectionKeys, maxAgeDays per entity type in the meta lifecycle).
    • constants() returns every config-derived value under its template name, computed from the loaded config on first call and cached until the config is reset. Importing the module reads nothing.
      • Paths (from paths()): JEEVES_BASE_DIR, CONFIG_DIR, CONTENT_DIR, SCRIPTS_DIR, CREDENTIALS_DIR, IMAP_SECRETS_DIR, GOG_CONFIG_DIR (paths().gogHome), TOKEN_METRICS_DIR. PIPELINE_CONFIG_PATH and SILO_ROUTING_CONFIG_PATH both name the loaded instance config file.
      • Derived from the content dir: GITHUB_DIR, GITHUB_REGISTRY_PATH, DEFAULT_MEETINGS_DIR, SLACK_DOMAIN_DIR; from the state dir: EMAIL_EVENTS_DIR; from the credentials dir: NOTION_API_KEY_PATH, X_OAUTH_DIR; from the gog home: GOG_CLIENT_PATH.
      • Integrations (from integrations()): INSTANCE_NAME, QDRANT_API_URL, QDRANT_SERVICE_NAME, GATEWAY_HOST, GATEWAY_PORT, GH_BIN, GH_CONFIG_DIR, GH_ACCOUNT, GH_BOT_USER, GOG_BIN, NOTION_VERSION, JIRA_SITE_URL, JIRA_EMAIL, JIRA_API_TOKEN_PATH, JIRA_FIELDS_FILENAME, JIRA_MAX_HISTORY, LINEAR_CONFIG_PATH, LINEAR_MAX_HISTORY.
      • X_ACCOUNTS: handle → content directory, each resolved through the account's silo (integrations.x.accounts.<handle>.silo, default silo) and relativePath (default x/<handle>).
      • Token metrics: TOKEN_RATES_PATH, TOKEN_RATES_PENDING_PATH, SLACK_DM_NAMES_CACHE_PATH (under TOKEN_METRICS_DIR), TOKEN_RATES_SEED_PATH (the seed core ships, config/token-rates.seed.json in the package).
      • OpenClaw: SESSIONS_DIR, OPENCLAW_AGENT_DB_PATH (agentDbPath('main'); agentDbPath(agentId, stateDir?) names any agent's DB, {stateDir}/agents/<agentId>/agent/openclaw-agent.sqlite, and rejects an id that isn't a plain name), CLAUDE_CODE_PROJECTS_DIR (under the OS home dir), OPENCLAW_UPGRADE_CUTOFF (env), SPAWN_WORKER_PATH (core's own spawn-worker module).

    Content written outside the configured content dir and silos is invisible to the watcher and jeeves-server, so every content path goes through paths() / siloPath() (Decision 28).

    Thin wrappers around date-fns. No config dependencies.

    • dayOfWeek(dateStr) — full weekday name (e.g., "Monday"). Important because LLMs cannot do day-of-week arithmetic reliably.
    • formatDate(dateStr, fmt) — format a date using date-fns pattern
    • relativeDays(dateStr, referenceStr?) — human-friendly relative description ("3 days ago", "today")
    • requireTimeZone(value, source) — validates a time zone read from instance config; throws (naming source) when it is empty or not a valid IANA zone. No default zone
    • withDateContext(task, now, timeZone) — prepends > **Today is <weekday>, <YYYY-MM-DD> (<zone>).** … to a worker task (used by taskFileDispatcher when given dateTimeZone)
    • Re-exports format and parseISO from date-fns

    From a shell in an instance repo:

    node --input-type=module -e "import { dayOfWeek } from '@karmaniverous/jeeves-scripts-core'; console.log(dayOfWeek('2026-06-01'));"
    

    Gmail parsing utilities for raw Gmail API payloads. No config dependencies.

    • headerValue(headers, name) — extract header value (case-insensitive)
    • extractTextFromPayload(payload) — recursively extract text and HTML from MIME parts, decoding base64
    • extractAttachments(payload) — recursively extract attachment metadata (filename, mimeType, size, attachmentId)

    GitHub CLI wrappers with typed invocation. Depends on GH_BIN, GH_CONFIG_DIR.

    • setupGhConfig() — sets GH_CONFIG_DIR env var so gh finds correct auth tokens
    • gh(args, options?) — run gh CLI command, return structured GhResult (ok, status, out, err). options: allowFail (return the failed result instead of throwing) and timeoutMs (kill gh after that long and throw gh <args> timed out after <n>ms; no limit by default)
    • ghJson(args, options?) — run gh and parse stdout as JSON
    • ghApi(endpoint, options?) — call GitHub REST API via gh api. Both take { timeoutMs } and pass it to gh

    Google Workspace CLI wrapper with retry. Depends on GOG_BIN, GOG_CONFIG_DIR.

    • gogWithRetry(args, opts?) — run gog command with retry logic for transient network errors (context deadline exceeded, timeouts). Defaults GOG_HOME to GOG_CONFIG_DIR (an existing GOG_HOME is kept), so gog and the scripts use the same home.

    Single source of truth for which gog credentials exist. Depends on GOG_CLIENT_PATH, GOG_CONFIG_DIR.

    • gogServiceAccountDirs(configDir?) — directories searched for service-account mailboxes, in order: <configDir>/data (current gog), then <configDir> (older gog without data/); gogServiceAccountDir(configDir?) is the first
    • serviceAccountFileName(email) — sa-<base64(email), padding stripped>.json, the file name findServiceAccountFile looks for in each directory
    • findServiceAccountFile(email, configDir?) — that mailbox's registration in the first directory that has it, or null
    • detectGogCredentials(configDir?) — { oauthClient, serviceAccount, any }: OAuth client file present, any sa-*.json present
    • requireGogCredentials(job, accountCount, creds?) — false when accountCount is 0 (caller skips), true when any credential exists, otherwise throws so the run fails

    Gateway HTTP client for OpenClaw tool invocation. Depends on GATEWAY_HOST (loopback) and GATEWAY_PORT (OpenClaw's own gateway.port, via gatewayPort()).

    • gatewayInvoke(tool, args, options?) — invoke an OpenClaw gateway HTTP API tool; the bearer token is gatewayToken() from openclaw-config.ts (CLAWDBOT_GATEWAY_TOKEN, else the OpenClaw config), and the call throws when there is none
    • unwrapResult(r) — unwrap result from gateway response

    Gateway RPC caller for methods that are not HTTP tools, or whose tool wrapper limits what the RPC allows. Depends on the global openclaw install (admin/lib/resolve-openclaw-dist.ts).

    • gatewayRpc(method, params, cliPath?, options?): run openclaw gateway call <method> --json --params <json> under the current Node binary (no shell) and resolve the result; gateway errors, CLI failures and non-JSON output reject. A gateway rejection (the CLI's { ok: false, error }) rejects with GatewayRpcError, which carries the gateway's code and details (e.g. { reason: 'session-changed' }) when it sent them. options.waitMs sets the CLI's --timeout (how long it waits for the gateway's answer; unset, the CLI's 10 s default)

    Reads the gateway port and credentials from the local OpenClaw config, ~/.openclaw/openclaw.json, then the legacy ~/.clawdbot/clawdbot.json (home: USERPROFILE, else the OS home dir). Files are JSON5, as OpenClaw reads them; only the fields read here are validated (openclawConfigSchema, Zod 4; gateway.port 1-65535). A missing file is skipped; a file that exists but can't be read, isn't JSON5 or is invalid throws, naming the file and the cause (#42). Token fields (channels.slack.botToken, accounts.<id>.botToken, gateway.auth.token) may be OpenClaw SecretRefs ({ source, provider, id }), as OpenClaw allows; core does not resolve them, so a SecretRef token (any object, whatever its source) reads as unset (CLAWDBOT_GATEWAY_TOKEN still supplies the gateway token). Env references are resolved against what core can see of OpenClaw's environment (the process env, then the .env beside the config file, then the config's env.vars, each filling only names still unset; the gateway's working-directory .env and login-shell import are not visible to core): ${VAR} anywhere in the string and a whole-string $VAR from the environment, $${VAR} as a literal; a reference to a missing or empty variable reads as unset, and the no-token error names it. Never logs a token. Exported from the package root.

    • gatewayPort(files?) — gateway.port, else OpenClaw's default 18789 (OPENCLAW_DEFAULT_GATEWAY_PORT)
    • gatewayToken(files?) — a non-empty CLAWDBOT_GATEWAY_TOKEN, else gateway.auth.token; null when neither exists
    • slackBotTokens(files?) — every Slack bot token by gateway account id (channels.slack.accounts.<id>.botToken, else the flat channels.slack.botToken as default), from the first file that has any; throws when none does
    • slackBotToken(accountId = 'default', files?) — one account's token; throws when it has none
    • findInOpenclawConfig(pick, files?) / openclawConfigPaths(home?) - the search primitives
    • openclawStateDirs(home?) - ~/.openclaw, then the legacy ~/.clawdbot; also used by constants() (SESSIONS_DIR, OPENCLAW_AGENT_DB_PATH) and spawn-worker's session lookup

    Used by gateway-client (and so spawn-worker), slack/poll and instance code.

    Another Jeeves component's settings belong in that component's config file, not in the instance config: code locates the file and reads from it. The layout is the platform's (@karmaniverous/jeeves): {configRoot}/jeeves-<name>/config.json, where the platform config root is paths().configDir. Exported from the package root.

    • componentConfigPath(name, configRoot?) — e.g. componentConfigPath('runner') is {configDir}/jeeves-runner/config.json
    • readComponentConfig(name, schema, file?) — the file validated with a Zod schema for the parts the caller reads; undefined when it does not exist; throws on bad JSON or a schema mismatch

    integrations().qdrant.apiUrl defaults to the watcher's vectorStore.url this way.

    The jeeves-runner's HTTP API address, from the runner's own config (validated with the runner's runnerConfigSchema; port defaults to 1937). Exported from the package root.

    • runnerConfig(file?) — the runner's config with defaults applied (schema defaults when the file does not exist)
    • runnerUrl(file?) — http://127.0.0.1:<port>
    • triggerRunnerJob(jobId, file?) — POST /jobs/<id>/run; resolves { status, body }, rejects when the runner is unreachable

    Used by admin/collect-token-metrics (to start admin-refresh-token-rates) and instance code that chains jobs.

    The jeeves-meta service's HTTP address, from the meta service's own config ({configDir}/jeeves-meta/config.json; only port is read, defaulting to the platform's META_PORT). Exported from the package root.

    • metaConfig(file?) - the meta config's port with the default applied (also when the file does not exist)
    • metaUrl(file?) - http://127.0.0.1:<port>

    Used by google-drive meta seeding (POST /seed).

    One report file per day, for runnables that keep a dated report (slack-refresh-channel-meta; instance runnables). Exported from the package root.

    • dateInZone(now, timeZone) - the calendar date in an IANA zone, YYYY-MM-DD
    • reportPath(outputDir, date) - <outputDir>/<date>.md, joined with the host's node:path
    • deliverReport(outputDir, date, report) - writes the report there atomically (atomicWrite from @karmaniverous/jeeves; directories created, a same-day report replaced, a final newline added when missing); returns the file

    Timestamped run directories for runnables that hand a worker a snapshot of files. Exported from the package root.

    • runId(now) - YYYYMMDD-HHMMSS in UTC, the run directory's name
    • pruneRuns(root, keep) - deletes all but the newest keep run directories under root (other entries are left alone; an unreadable root deletes nothing); returns what it deleted

    A file's jeeves-server insider link, resolved by the server itself, so it works on any host and never posts a filesystem path. resolvePublicUrl, slackLink, liveReportLink and linkBeforeWorker (and the ServerAccess, Fetch, FetchResponse, ReportLink and LinkRequest types) are exported from the package root, so an instance job that writes a dated report links it the same way (Decision 74); the rest serves slack-refresh-channel-meta.

    • resolvePublicUrl(fetch, access, fsPath, timeoutMs?) - GET <apiUrl>/api/resolve-path?fsPath=<path>&k=<insiderKey>; resolves the server's publicUrl. Throws jeeves-server could not resolve <path> (HTTP <status>[: <error>]) on a non-OK answer (e.g. a path under no root) and jeeves-server publicUrl is not configured; refusing to post a filesystem path to Slack when the server has no publicUrl. It waits at most timeoutMs (default SERVER_LINK_TIMEOUT_MS, 10 s; the server is local, so a longer wait means it is hung) for the answer, body included, then throws jeeves-server did not answer within <ms> ms resolving <path> (never the URL or key)
    • slackLink(url, label) - <url|label>, a Slack mrkdwn link
    • serverAccessFrom(config, keyName) / readServerAccess(keyName) (on the root export, with the ServerLinkConfig type) - { apiUrl: http://127.0.0.1:<port>, insiderKey } from jeeves-server's own config (readComponentConfig('server', ...)): the insider key is computeInsiderKey(seed) from @karmaniverous/jeeves (Decision 72) over the named machine key in keys (a seed string or { key }). Throws naming a missing config, port or key
    • liveReportLink(title) and linkBeforeWorker(...) - a report's link (<url|<title> — <YYYY-MM-DD>>) resolved before a worker runs: a live run fails when it can't be resolved, a preview logs a [warn] and shows (no link)

    Recovers an LLM worker's full final reply after dispatchSession: it takes the session key from spawn-worker's WORKER_RESULT line and reads the last assistant message via the gateway chat.history RPC with maxChars: 500000 (the sessions_history tool caps text at 4000 characters, which cut long replies). A reply the gateway still marks as truncated fails with worker reply truncated by gateway. Job scripts use it to verify structured worker results instead of trusting the exit code.

    Job-side Slack I/O for LLM workers. On OpenClaw 2026.9, sub-agent sessions have no message tool, so the job script does all Slack work:

    • worker-slack-config.ts: Zod 4 schema for the job's { accountId?, reads?, posts? } config (each post target carries editTs?, the exact message ids the worker may edit, and pin?); types are derived with z.infer.
    • run.ts: dispatchWithSlack(task, dispatchOptions, { reads, posts }) is the production entry point. It validates the config before any gateway call and supports --dry-run (report the job, TASK length and post targets; no dispatch, no posts) and --print-task (print the TASK, no dispatch).
    • worker-slack-job.ts: orchestration with injected deps (read → TASK → dispatch → validate → post/pin/edit).
    • worker-posts.ts: the slack-posts output contract (a fenced JSON array, never a bare object, of {channel, text, thread_ts?, pin?, edit_ts?}; edit_ts replaces the text of an existing message and can't be combined with thread_ts/pin; edits and pins are allowed only where the target's editTs / pin permit), the worker instructions, and the Slack context formatting (fenced as untrusted data the worker must never follow as instructions).
    • slack-io.ts: read / send / pin / edit through the gateway message tool (/tools/invoke). A read response without a valid messages array throws (only messages: [] means an empty channel).
    • slack-target.ts: normalizes Slack IDs to channel:… / user:… targets (the prefix must match the ID family: channel: C/G/D, user: U/W). Exported from the package root as normalizeSlackTarget, so an instance dispatcher that takes a target from its own config normalizes it the way core does.

    Gateway session spawner: the executable runDispatcher() / dispatchSession() run (constants().SPAWN_WORKER_PATH, core's built dist/lib/spawn-worker.js). It uses gateway-client, so it reads the gateway port and token from the OpenClaw config.

    Usage: echo "task" | node <core>/dist/lib/spawn-worker.js --job-id=<id> [--label=<label>] [--thinking=<level>]

    • Spawns a session via OpenClaw gateway HTTP API
    • Polls indefinitely for completion (runner job timeout_seconds handles process kill)
    • Waits for transcript to flush
    • Outputs WORKER_RESULT:{"sessionKey":"...","tokens":12345,"durationMs":123000} on last stdout line
    • Implements retry with exponential backoff (3 retries, 30s base)

    YAML frontmatter for generated Markdown: builds the --- block (every value JSON-quoted, so no value can break it) that the converters and the Google Drive sync put at the top of a file.

    Waits for a spawned worker: polls the gateway's sessions_list for the child session until its run ends, then reads token usage from the transcript (waitForWorkerCompletion, findSession, sessionTokens). spawn-worker.ts uses it.

    getTokensFromTranscript(path) sums message.usage.totalTokens over a session transcript's JSON lines (0 for a missing or unreadable file, which it logs). worker-session.ts uses it.

    Core has no retry module of its own (#24): it imports the platform's from @karmaniverous/jeeves and keeps the same names on its root export. One difference from the old copy: policy.attempts must be an integer of at least 1, or the helper throws RangeError before calling fn (the old copy ran fn once). retryAsync(fn, policy) and retrySync(fn, policy) try up to policy.attempts times, waiting policy.delayMs(attempt) (exponentialBackoff(baseMs) for doubling waits) after a failure policy.isRetryable accepts, and rethrow the last error. The worker spawn, the Slack client's 429 handling, the Drive rename, the Fathom share fetch and the Notion page reader use it.

    Validated JSON file I/O, on the root export: readJsonFile(file, schema) parses and Zod-validates (throwing), readJsonFileIfExists(file, schema, fallback) returns fallback for a missing file, and writeJsonFile(file, data, indent = 2) writes atomically (temp file, rename), creating parent directories. (@karmaniverous/jeeves's readJson / writeJsonAtomic remain for fallback reads and fixed-indent writes.)

    The one argument parser for runnables, core's and instances', on the root export (Decision 59). Core has no copy of its own (#24, jeeves#151): it imports the platform's from @karmaniverous/jeeves (0.8.0 and later) and keeps the same names on its root export, unchanged in behaviour: argFlag(name, description?), argValue(name, fallback?, { alias?, description? }) and argOptional(name, { alias?, description? }) build the spec a runnable declares as defineRunnable({ flags }); run checks the arguments against it with unknownArgs(args, spec, positionals?) and the job reads ctx.flags (parseRunnableArgs(args, spec)), plus positionalArgs(ctx.args, spec?). The convention (bare flags, --name value and --name=value, positionals) and the checks are in runnables.md.

    isRecord(v): the one type guard for a non-null object read from unknown JSON, on the root export (arrays pass; admin/lib/openrouter-pricing.ts keeps its own, which rejects arrays).

    dispatcherFlags (--print-task, --dry-run), for a dispatcher's defineRunnable({ flags }), on the root export. resolveDispatcherFlags reads them from parsed flags or, for dispatchers that declare none, from raw arguments (default process.argv).

    The one Slack Web API client, on the root export (Decision 59): createSlackClient({ token, fetch?, rateLimitAttempts?, sleep? }). Core's Slack poller reads through it (slack/lib/slack-api.ts, slack/lib/file-content.ts). Usage, errors and rate limits are in slack.md.

    Slack's three control characters (&, <, >) both ways: escapeSlack(s) for text a job posts, so <!channel> or <@U123> in it is shown, not acted on; decodeSlack(s) for text Slack returns. Both are on the root export. Used by slack/channel-meta and admin/skill-review.

    Shared entity persistence — upsert, backfill, and delete entity files with reverse-diff history using fast-json-patch. Used by any domain that persists structured entities (Jira, Linear, etc.).

    • upsertEntity(domainDir, type, key, current, now, maxHistory) — create or update entity file with reverse-diff history; returns file path
    • backfillEntity(domainDir, type, key, current, now) — write entity file only if it doesn't exist (historical import); returns path or null
    • deleteEntity(domainDir, type, key) — delete entity file; returns boolean
    • writeUnmatched(domainDir, label, body) — write unrecognised webhook payload to _unmatched/ subdirectory
    • readStdinJson() — read stdin to completion and parse as JSON; used by Event Gateway drain scripts (on the root export)

    Entity file structure:

    {
    "entityType": "issue",
    "entityKey": "ACME-1",
    "current": { ... },
    "history": [{ "ts": "...", "patch": [...] }],
    "meta": { "firstSeen": "...", "lastWebhook": "...", "lastBackfill": null, "version": 7 }
    }

    History entries are reverse-diff patches (JSON Patch format) — apply newest-to-oldest to reconstruct prior states.

    The instance config (pipeline accounts, buckets, refs, email and Drive settings, silo routing) is documented in config.md, including the pipeline block and silos.