Write Changelogs From GitHub Commits With AI

Release automation should remove reconstruction work, not remove judgment. In this lesson, we use a personal fork of the public, source-available Keystroke repository as a real source instead of staging a toy project. We connect the fork through Userorbit's GitHub integration, index the useful repository context, configure the Release Agent to suggest rather than publish, and run a 30-day historical backfill. The automation turns that activity into multiple changelog drafts. Future weekly runs can use a 7-day lookback and surface only the strongest release signals. The result is a review queue grounded in what changed, not an unattended stream of official release notes.

Start with a real repository you can safely connect

The recording begins in a dedicated demo workspace. The upstream Keystroke repository has already been forked into a personal GitHub account, which keeps the demonstration isolated while preserving real commit history.

In Userorbit:

1. Open Integrations.
2. Create a GitHub connection.
3. Authorize Userorbit for the account that owns the fork.
4. Select the repository and save the connection.

For your own product, connect the repository that contains the customer-facing changes you want the release workflow to understand. You do not need to fork a repository in normal use; the fork is useful here because this is a public demonstration using someone else's project.

Index signal, not every file

After the connection is active, enable indexing for the repository. This is where you decide what the release agent can use as context.

Include the parts of the codebase that explain product behavior. Exclude generated output, dependencies, fixtures, and other paths that create volume without explaining a customer-visible change. The demonstration caps the first pass at 100 files so the setup remains quick and observable.

This boundary matters. A release agent cannot distinguish a meaningful product change from repository noise if both are presented with equal weight.

Keep the Release Agent draft-first

While the first index runs, open the AI settings and review the Release Agent. In the recording, its operation mode is suggest only. It can propose changelog entries, but it cannot publish them directly.

The workspace already gives the agent general release-note guidance. Add custom guidance only when the output needs to reflect your product vocabulary, brand voice, audience, or editorial rules. Useful instructions include:

• which changes deserve their own announcement
• which internal terms should become customer language
• how to describe availability and plan limits
• which claims require a link, screenshot, or manual confirmation
• which categories should never be published without approval

The automation supplies speed. Suggest-only mode preserves accountability.

Backfill once, then follow the shipping cadence

Once indexing finishes, create the release automation. The demonstration uses a weekly automation scheduled for Monday.

A weekly schedule normally uses a 7-day lookback. For the first run, the lookback is expanded to 30 days so the automation can reconstruct a useful historical period and demonstrate how it handles several release signals at once.

Select the indexed repository, add any case-specific drafting instructions, choose the destination category, and save the automation. Run it manually once before relying on the schedule. The first run should answer three questions:

1. Did the agent find the changes that mattered?
2. Did it ignore low-value repository activity?
3. Are the drafts accurate enough to review quickly?

Review the generated changelog proposals

The 30-day run creates several drafts because it covers more activity than a normal weekly window. One proposal identifies the project's use of custom MCP as an agent tool. Opening a draft shows the generated explanation and the source context behind it.

Treat this screen as a release editor's queue, not a finished changelog. For every proposal:

1. Confirm the change exists in the selected period.
2. Check that it affects users rather than only internal implementation.
3. Rewrite the headline around the user outcome.
4. Verify availability, plan, platform, and release status.
5. Add a real screenshot when the interface changed.
6. Merge related drafts and discard weak or duplicate signals.
7. Publish only after a human owns the final claim.

Later weekly runs use the shorter lookback, so the agent has fewer changes to compare and can concentrate on the highest-value items.

Release automation setup checklist

SOURCE
[ ] Connect a repository you are authorized to read.
[ ] Confirm the integration cannot expose unrelated private repositories.
[ ] Include product code and customer-facing context.
[ ] Exclude dependencies, generated output, fixtures, and repository noise.

AGENT
[ ] Keep the Release Agent in suggest-only mode for the first runs.
[ ] Add brand language, audience, availability, and evidence rules as guidance.
[ ] Define what qualifies as a customer-visible release signal.

AUTOMATION
[ ] Match the recurring lookback to the schedule (7 days for a weekly run).
[ ] Use a longer lookback only for an intentional first backfill.
[ ] Select the destination category and run the automation manually once.

REVIEW
[ ] Verify every proposal against its source commit, PR, or release.
[ ] Merge duplicates and discard internal-only changes.
[ ] Confirm plan, platform, status, links, and screenshots.
[ ] Require a human decision before publishing.

Transcript

Lightly edited for clarity.

I've created a demo workspace for Keystroke. First, we'll connect the source for the release automation. I go to Integrations and create a new connection. I've already forked the repository into my personal GitHub account, so I select that fork and save it. GitHub asks me to authorize the connection.

Once the repository is connected, we can enable indexing. Here we declare what to include and exclude. That boundary can improve the quality of the output because the agent sees more of the useful product context and less repository noise. For this test, I'll cap the first pass at 100 files and start indexing. The first index can take a couple of minutes.

While it runs, we can open the AI settings and inspect the agent. This is the Release Agent configured for the workspace. Its operation mode is suggest only, so it will not publish anything directly.

There are also advanced settings and guidance. Guidance lets you add custom instructions that the agent should use when it runs the workflow. A new workspace already includes general knowledge about release-note structure, best practices, and technical writing. If you want the output to reflect your brand voice or product terminology, you can add that as guidance for the agent to apply on top of the default instructions.

The indexing has finished, so now we'll create the release automation. I'll call this a weekly Keystroke release-signal automation and schedule it for every Monday. A weekly schedule would normally use a 7-day lookback. For the first run, I'll change that to 30 days so we can backfill a historical period.

Next, select the repository. Add any drafting instructions that are useful for this specific case, choose the category where the drafts should go, and create the automation.

I'll run it manually for the first time. The run takes a couple of minutes. When it completes, we start seeing AI-generated announcement drafts.

The automation has begun creating proposals. One of them identifies the use of custom MCP as an agent tool. Because the first run covers 30 days, it creates several items in one pass. When the automation runs on the shorter weekly schedule, it has a smaller set of changes to evaluate and can focus on the higher-value release signals.

Opening a proposal shows the generated explanation. You can also extend the workflow so the agent captures screenshots from the application when the integration is configured for that. The important part is that these are drafts for review. The automation reconstructs the release context; a person still decides what is accurate, useful, and ready to publish.

Release automation FAQ