Automating GitHub Workflows with DynaCI

Introduction to DynaCI

For teams in banking, education, healthtech, fintech and tax returns, CI pipelines are not just convenience. They are compliance artifacts. A missed test step, a hardcoded secret, or an undocumented deployment path can translate into audit findings, data handling violations, or delayed releases in environments where change control is tightly governed.

DynaCI is a system for generating GitHub Actions workflows through dynamic analysis. The problem it solves is the gap between what developers actually run locally and what is captured in static repository history or manual templates. That gap causes missing dependencies, incorrect test commands, and environment drift that require manual repairs.

DynaCI is evaluated on 50 popular Python projects that use GitHub Actions, a popular CI platform, and compared against the state-of-the-art CI workflow generation approaches: GH-WCOM and LLM4GW. DynaCI outperforms these existing approaches by more than 21.13% in BLEU score and 20.96% in F1 score on average [1].

The relevance for regulated sectors is straightforward: the same pressure to automate safely exists whether you are shipping a lending API, a student records system, a patient portal, or a tax filing service. Accuracy and fewer manual repairs matter more than speed alone when a workflow touches production data or regulated processes.

How DynaCI Works

High-level overview: DynaCI starts from observed local runs, builds a first draft from that evidence, then refines the draft with an LLM.

Technical steps:

• trace the developers' local environment to collect runtime information
• construct an initial workflow by applying runtime-informed heuristics
• iteratively refine the workflow via LLM-based Retrieval Augmented Generation (RAG) [1]

DynaCI uses runtime information from developers' local environments to generate CI workflows [1]. DynaCI uses LLM-based Retrieval Augmented Generation (RAG) to refine workflows [1].

The pipeline can be summarized as:

By grounding the first draft in what actually runs locally, DynaCI reduces the gap between assumed and real dependencies, test commands, and environment variables.

A minimal illustration of the kind of output DynaCI aims to produce is:

name: CI
on: [push, pull_request]
jobs:
  test:
  runs-on: ubuntu-latest
  steps:
  - uses: actions/checkout@v4
  - name: Setup Python
  uses: actions/setup-python@v5
  with:
  python-version: '3.11'
  - name: Install deps
  run: pip install -r requirements.txt
  - name: Run tests
  run: pytest

GitHub Actions can be used to build out more advanced, custom workflows for anything from triggering an alarm to orchestrating complex security test automations [2]. Secrets on GitHub are used to securely store sensitive data such as passwords, tokens, and certificates, among other things—and they can be directly referenced in workflows, too [2]. GitHub Actions supports sharing data between jobs in any workflow as artifacts, which are associated with the workflow run where they are created [2].

Benefits of DynaCI

Accuracy gains. The reported outperformance of more than 21.13% in BLEU score and 20.96% in F1 score on average suggests better alignment with existing, human-written workflows [1].

Fewer manual repairs. It also generates workflows without missing functionality in 56% of cases, while requiring fewer manual modifications to repair such exceptions [1].

Runtime grounding. Using local runtime traces to collect information helps capture implicit steps that are not documented in README files [1].

Limitations of DynaCI

Coverage is not complete. Workflows are generated without missing functionality in 56% of cases, implying manual review remains necessary for the remaining cases [1].

Language scope. The evaluation is Python-focused. The reported results are on 50 popular Python projects that use GitHub Actions [1]. Generalization to other languages, build systems, or highly custom compliance steps requires validation.

Operational considerations. Tracing developers' local environments to collect runtime information [1] raises questions about data minimization, PII exposure, and reproducibility in banking and healthtech settings where local machines may process sensitive data.

In education and tax return software, where seasonal release windows are narrow and audit trails must be clear, the reduction in manual modifications is valuable, but the 44% exception rate means teams still need a review gate before merging generated workflows.

Potential Applications in DevOps and Software Development

Banking and fintech. Generate CI jobs that run unit tests, static analysis, and security scans, with workflow definitions versioned in Git. The ability to share data between jobs as artifacts [2] supports multi-stage pipelines where test results are packaged for compliance reporting.

Healthtech. Runtime-informed heuristics can help surface the exact commands used in a developer's validated environment, reducing drift between local and CI. Any generated workflow still needs review to ensure no PHI is logged or exposed via secrets [2].

Education technology. Small teams can automate CI for multiple services to reduce toil, with release cycles aligned to academic terms.

Tax return platforms. Strict annual deadlines and regulatory testing requirements benefit from automated drafts that still require human sign-off for auditability.

Across these contexts, automating CI generation saves engineer time, but accuracy and auditability remain non-negotiable.

Comparison with Other Tools and Platforms

RepoKitteh

RepoKitteh is a tool that generates GitHub Actions Workflows via dynamic analysis [3]. RepoKitteh is powered by Starlark, a proven Python dialect [3]. GitHub API accessible as Starlark functions [3]. For that reason we offer two flavors of the app - one that has only read only content permissions and one that requires read & write permissions [3].

Lacquer

Lacquer emphasizes developer workflow ergonomics:

• Your workflows are just YAML files. Commit them, review them, version them like any other code [4].
• Test everything on your laptop before deploying. No cloud account needed [4].
• Build reusable workflow components that enforce consistent operational procedures across teams and environments [4].
• Run steps conditionally based on the output of previous steps or break out steps into sub steps which run until a condition is met [4].

Keep

Keep focuses on monitoring automation:

• Keep is GitHub Actions for your monitoring tools [5].
• A Keep Workflow is a declarative YAML file that automates your alert and incident management [5].
• Each workflow consists of: Triggers, Steps, and Actions [5].
• Flexible Deployment - Deploy on-premises or in air-gapped environments with cloud-agnostic architecture [5].

TraceRoot

TraceRoot, the platform family associated with dynamic tracing research, offers deployment options:

• Create an account https://app.traceroot.ai to get started without running the platform yourself [6].
• Native SDKs for Python and TypeScript are supported [6].
• See the self-hosting guide for configuration and deployment details [6].

High-level comparison

Tool	Core Mechanism	Workflow Representation	Notable Constraint
DynaCI	Runtime tracing + heuristics + LLM RAG [1]	GitHub Actions YAML	Evaluated on Python projects; 56% fully functional without edits [1]
RepoKitteh	Dynamic analysis via Starlark [3]	GitHub Actions via Starlark functions [3]	Two permission flavors: read-only vs read/write [3]
Lacquer	Local-first GitOps native workflows [4]	YAML, versioned like code [4]	Emphasis on composable steps and complex control flow [4]
Keep	Declarative monitoring workflows [5]	YAML with triggers/steps/actions [5]	Monitoring focused, on-prem/air-gapped deployment [5]

For banking, healthtech and tax teams, the choice often hinges on governance. DynaCI offers higher reported generation accuracy than existing approaches [1] and fewer manual repairs [1], which reduces review burden. RepoKitteh provides explicit permission models for sensitive repositories [3]. Lacquer provides GitOps-native versioning [4] and local testing without cloud accounts [4], which aligns with air-gapped policies. Keep addresses post-deployment operational automation rather than CI generation.

In practice, many regulated organizations use a combination: runtime-informed generation for initial drafts, GitOps review for change control, and strict secret handling in GitHub Actions [2] for production safety.

Conclusion

DynaCI addresses the problem of missing or inaccurate CI steps by grounding workflow generation in local runtime evidence and refining it with LLM RAG. The evaluation shows measurable accuracy gains and fewer manual repairs on Python projects, with 56% of workflows generated without missing functionality [1]. Limitations around language scope, coverage, and tracing operational concerns mean human review remains essential in regulated contexts. Combined with GitOps practices and careful secret management, it can reduce toil while preserving auditability for banking, healthtech, education, and tax software teams.

References

1. DynaCI: Generating GitHub Actions Workflows via Dynamic Analysis — arxiv.org
2. 7 advanced workflow automation features with GitHub Actions — github.blog (18 November 2021)
3. RepoKitteh: Repository Functions — repokitteh.io (2 April 2021)
4. lacquerai/lacquer — github.com (22 August 2025)
5. keephq/keep — github.com (2 November 2026)
6. traceroot-ai/traceroot — github.com (2 November 2026)