Self-Maintaining Software: How Our Platform Keeps Itself Alive

Every morning at 06:00 UTC, our platform checks itself. It reads its own alarms, pings its own agents, looks at what got stuck overnight, and writes a short report. If something is wrong, it fixes what it is allowed to fix and opens a GitHub issue for the rest. By the time we read the issue, another agent has usually already opened the pull request.

We did not set out to build a self-maintaining system. We set out to use our own product the way we ask customers to use it. This is what fell out.

Maintenance Is a Loop, Not a Backlog

Most operational work is the same handful of questions asked every day. Is anything firing? Is the database running out of room? Did last night’s job fail? Did a dependency get a CVE? Is a task waiting on a human who went home?

None of these need judgment to ask. They need judgment to answer. So we split the job in two: a schedule that asks, and an intelligence that answers.

StratoNext is the schedule. It holds the projects, the environments, the signals, the jobs, the approvals, and the audit trail. Every piece of work is a task, and every task has a result. Nothing happens off the record.

The intelligence is an agent. Today it is Claude Code running with a StratoNext agent identity. It picks up tasks, reads the platform through the CLI, reads the cloud through brokered credentials, and reports back. It never holds a standing key. It never approves itself.

Signals Carry the Facts

We do not ask the agent to go rummage through CloudWatch. We register what matters as signals on the deployment, with thresholds, and the platform reads them live.

stratonext signal list --project-id prj_platform --status
API ECS CPU utilization        OK        3.1 %   warn 60  crit 80
API healthy targets            OK        1       crit <1
RDS free storage               OK        15.2 GB warn 4  crit 2
Job firing DLQ messages        OK        0       crit >0
Frontend CloudFront 5xx rate   OK        0.00 %  warn 1

Twenty-nine signals cover our production deployment. Half of them are alarms we already had. The rest are plain metrics with a warning and a critical threshold set on them in StratoNext. No alarm resource to create, no extra cost, and the platform reports their state the same way. The agent reads one list and knows the state of the system. It spends its tokens on the anomalies, not on the inventory.

StratoNext project page showing signal counts by state, key numbers with sparklines, and a firing alarm on the production deployment — Sample view of the signals of a project.

Signals also put a layer between the agent and the monitoring stack. The source can be CloudWatch today, Datadog or Sentry tomorrow, and the agent sees the same list either way: a name, a value, a state. We decide at the project level what counts as a key metric, and that is what the agent watches. It needs no keys to the monitoring platform and no knowledge of its API. When a signal turns red and the quick read is not enough, the agent goes to the monitoring system directly, outside StratoNext, through the vendor’s MCP server, and digs into logs, traces or dashboards there.

Signals are not a monitoring solution and do not try to be one. They are the control panel a project manager would want: a short, curated view of whether the system is healthy, with the detail one step away when it matters.

The Heartbeat Job

The daily self-check is a scheduled job assigned to an agent. The instructions are a page long and read-only.

stratonext job create --target PROJECT --project-id prj_platform \
  --agent-id agt_ops --schedule "cron(0 6 * * ? *)" \
  --name "Daily self-check" --instructions "$(cat self-check.md)"

It runs five checks. Agents not seen in 48 hours. Environments that fail verification. Signals not in OK. Job runs whose task failed or is still open after 12 hours. Tasks waiting on input or approval for more than a day.

The output is a status, GREEN, AMBER or RED, and a report under 60 lines. On GREEN it writes the report and completes. On anything else it creates one follow-up task for the ops agent, with the flagged items and nothing else. That task does the fixing. A human hears about it only if the agent runs out of allowed actions and escalates.

The rule that makes this bearable is simple. The job is allowed to create exactly one task per run. A noisy self-check is worse than no self-check.

Alarms Become Tasks

Scheduled checks catch slow problems. Alarms catch fast ones. We wired both into the same place.

Every production alarm publishes to an SNS topic. The topic calls a StratoNext webhook. The webhook creates a task for the ops agent to investigate that one alarm. The alarm payload travels with the task as data, so the agent starts from the exact metric, threshold and time it fired, not from a generic health check.

The agent picks the task up within a minute, diagnoses, and acts within what the environment allows. Every environment carries an autonomy policy, and ours is deliberately set: production auto-approves up to medium risk, anything above waits for a person. A restart of an ECS service passes. A change to a security group does not. The agent requests the permission, the task pauses waiting for approval, and one of us approves or rejects from the phone.

When the task closes, the agent writes an incident report and links it to the project.

GitHub Closes the Loop

Operations finds problems. Code fixes them. The hand-off between the two is where most teams lose a week.

When the ops agent finds something that needs a code change, StratoNext opens a GitHub issue on the repository linked to the project. The issue carries what the agent found: the signal, the evidence, the suspected cause.

From there the coding agents take over, on GitHub, the way they would for an issue a person wrote. One picks the issue up and proposes a fix in a pull request. When the fix needs more than the issue text, the agent uses the StratoNext CLI to look at the environment: current signal values, recent task results, the live state of the deployment. It gets operational, real-time context without anyone pasting it into the issue, and without holding cloud keys of its own.

The Loop Watches the Loop

A self-maintaining system fails quietly when the maintenance itself breaks. So one more weekly job audits the machinery: job runs that failed, agents that went stale, tasks stuck for more than a day, signals that stopped reporting data, webhooks that stopped firing.

That job files a task for a human. It is the only task we are not allowed to delegate.

A Second Instance Holds the Watch

There is one failure the loop cannot catch from the inside. If the platform that schedules the self-check is the thing that is down, no job fires, no task is created, and nobody is told. The system that is supposed to raise the alarm is the one that needs it. That is a deadlock, and it is the first thing anyone asks about when we describe this setup. It is also a problem only we have. Our customers do not need a second instance. One StratoNext is enough to keep their software alive. StratoNext maintaining StratoNext is the special case, and the second instance below is how we handle it.

So the main instance is not alone. A second, smaller StratoNext instance runs in a separate account and region, with its own agent and its own brokered access to the main account. It is the same platform running the same loop, pointed at the primary. It has exactly one project: the main instance. Its signals are the main instance’s external health checks, API latency and job firing lag.

The secondary has two ways in. When one of those signals fires, the alarm, which lives in the cloud account and not in the platform, calls a webhook on the secondary, and the secondary wakes up and creates a task for its agent to investigate that alarm. Independently of any alarm, the secondary’s own daily self-check looks at the main instance from the outside, so a quiet failure is caught within a day even if no alarm ever fires.

Either way the task runs the same ladder. The agent tries to fix it first, within what the environment allows: restart the API service, roll the deployment back. If the cause is in the code, it opens the issue and the fix comes as a pull request. Only when none of that brings the primary back does the task ask a human for help. The secondary never does anything else. It does not run our workloads, it does not hold our data, and it is cheap enough to forget about.

The primary watches the secondary the same way, with one signal and one webhook. Two small watchers that cannot both be down for the same reason are worth more than one large watcher that can.

What We Learned Running It

The agent is replaceable. The record is not. We have swapped the model behind the ops agent twice. Nothing else changed, because every task, approval and report lives in StratoNext, not in a chat history.

Policy beats prompting. “Do not touch production” in an instruction is a hope. An environment that caps the agent’s IAM role is a guarantee. We lean on the second.

Read-only agents earn trust fast. Our first three months were reports only. The agents got write access to production, capped at medium risk, after we stopped reading the reports because they were always right.

What Comes Next

Today the intelligence runs beside the platform. An external agent holds the model, StratoNext holds the schedule and the record. It works, and it is how we run the company. It also means every customer has to bring their own agent to get the same loop.

In the next release the intelligent layer moves into StratoNext itself. The self-check, the alarm triage and the issue filing become platform agents you enable on a project, with the same approvals, the same environment caps and the same audit trail you already have. You bring the cloud account and the repository. The platform brings the loop.

We have been running it on ourselves first. That is the whole point.