CyborgShell - An Integrated Orchestrator Built on DAG Principles

Table of Contents

What is a DAG?

A Directed Acyclic Graph (DAG) is a structure for organizing work: nodes (tasks) connected by directed edges (dependencies), with one critical rule: no cycles in the graph structure.

The DAG Rule: You cannot have A→B→C→A. The graph flows in one direction only.

Why this matters: It guarantees parallelization is possible, prevents circular dependencies, and makes the system understandable.

This acyclic constraint does NOT prevent iteration, looping, or feedback-based workflows. It only prevents cycles IN THE GRAPH STRUCTURE. You can re-execute tasks, trigger them based on conditions, and feed outputs back as new inputs.

Traditional DAG Tools vs CyborgShell

The key architectural difference: Traditional DAG tools separate the DAG from the orchestrator. CyborgShell unifies them.

Traditional Architecture (Airflow, dbt)

DAG Definition (YAML/Python)
        ↓
Orchestrator (Airflow Scheduler, dbt CLI)
        ↓
Executes DAG, handles retries, scheduling, etc.

You write config to tell orchestrator what to do.
Orchestration logic lives separate from DAG.

CyborgShell Architecture

Files (nodes) + Links (edges) = DAG Definition
        ↓
CyborgShell IS the Orchestrator
        ↓
CyborgShell executes, monitors, cascades, loops, gates

You define orchestration through links and transformers.
Orchestration and DAG are unified.

How CyborgShell Works

CyborgShell is an integrated orchestration platform where:

Because CyborgShell IS the orchestrator, it actively manages the DAG. It detects when files change, automatically cascades downstream, applies quality gates, and routes data through feedback loops.

Live Example: 2 Minutes

Watch what happens in real time:

# SETUP (30 seconds):
newfile 10
files
file 1
edit 10
10 write me a children's story about a donkey

link 2 1 chatgpt openai treat all as a prompt
# Instant: File 2 auto-processes (no manual trigger)

link 3 2 translate openai german
# Instant: File 3 auto-processes (no manual trigger)

link 4 3,2 chatgpt openai word count and top 10 words
# Instant: File 4 auto-processes with multi-input

project save story-generator

# Result: DAG created, all files processed automatically


# FIRST RUN:
files
# File 1: D (dirty - you edited it)
# File 2: D (1699 bytes, story generated)
# File 3: D (1960 bytes, translated to German)
# File 4: D (304 bytes, word analysis of both)


# ITERATE - Edit original file (5 seconds):
file 1
edit 10
10 write me a children's story about a donkey

# CyborgShell detects file 1 is dirty
# ALL transformers cascade automatically
# File 1 → File 2 → File 3 → File 4

files
# File 1: D (dirty)
# File 2: D (1506 bytes - regenerated)
# File 3: D (1682 bytes - retranslated)
# File 4: D (326 bytes - reanalyzed)


# KEY INSIGHTS:
# - Add new transformer = auto-processes (no manual trigger needed)
# - Edit original file = all downstream auto-reprocess instantly
# - CyborgShell detects dirty files and cascades automatically
# - This IS active orchestration, not passive DAG execution

Real-World Examples

Example 1: Document Processor with Automatic Feeding

newfile 5
files
file 1
filename trigger.txt

file 2
filename current-document.txt

file 3
filename processed-document.txt

file 4
filename results.txt

file 5
filename archive.txt

# DAG (acyclic): 1→2→3→(4,5) then 4→1 (feedback loop)
link 2 1 documentfeeder     # Feeds docs from inbox automatically
link 3 2 chatgpt claude "analyze and summarize"
link 4 3 chatgpt openai "extract insights"
link 5 3 passthrough        # Archive
link 1 4 trigger            # Writes "GO" to file 1, triggering next cycle

project save document-processor

# What CyborgShell's orchestrator does:
# - File 1 starts with "GO" (or gets it from trigger)
# - documentfeeder pulls first document from inbox/mailbox
# - Cascade: 1→2→3→(4,5)
# - trigger writes "GO" to file 1, marks it dirty
# - CyborgShell auto-cascades, documentfeeder pulls next document
# - Loop continues until inbox empty (documentfeeder returns IDLE)
# - Self-regulating infinite pipeline
# - New documents trigger processing instantly

Example 2: Statistical Analysis with Adaptive Looping

project load stat-analysis
files
# DAG structure (acyclic): 1→2→(3,4,5,6)→(7,8,9)→10→11→12→13→14
# Multiple branches, merge points, quality checks

# What CyborgShell's orchestrator does:
# 1. Edit file 1 with experiment parameters
# 2. Cascade: 1→2→(parallel: 3,4,5,6)→(parallel: 7,8,9)→10→11→12→13→14
# 3. File 13: quality gate checks if score >= 8 output PASS, else output STOP
# 4. If score too low:
#    - Edit file 3 (statistical methodology)
#    - CyborgShell reruns: 3→7→10→11→12→13→14
#    - Quality reassessed
# 5. If score still low:
#    - Edit file 2 (data generation)
#    - CyborgShell reruns entire cascade
# 6. Loop until quality threshold met

# The DAG structure never changes (always acyclic)
# But CyborgShell routes data through loops based on quality gates

Example 3: AI Coding Assistant with Recursive Refinement

newfile 5
files
file 1
filename complex-task.txt

file 2
filename breakdown.txt

file 3
filename solutions.txt

file 4
filename quality-check.txt

file 5
filename final-code.txt

# DAG (acyclic): 1→2→3→4→5
link 2 1 chatgpt claude "break this into N subtasks"
link 3 2 chatgpt openai "solve each subtask"
link 4 3 chatgpt claude "rate code quality 1-10"
link 5 4 chatgpt claude "if quality >= 8 pass code through, else output STOP"
link 6 5 blocker  # Blocks if quality check returns STOP

project save adaptive-coder

# What CyborgShell's orchestrator does:
# 1. Edit file 1: "Build a weather app with real-time updates"
# 2. Cascade: 1→2→3→4→5
# 3. Quality score: 6 (blocked)
# 4. Edit file 2: "Break into 20 subtasks instead of 10"
# 5. Cascade: 2→3→4→5 (quality reassessed)
# 6. Quality score: 7 (still blocked)
# 7. Edit file 2: "Break into 30 micro-subtasks"
# 8. Cascade: 2→3→4→5
# 9. Quality score: 9 (passes, file 5 processes)

# Same DAG, but orchestrator routes through feedback loop
# Each iteration gets finer granularity until quality meets threshold

Why This Architecture Matters

Aspect Traditional DAG Tools CyborgShell
Architecture DAG + Orchestrator (separate) Unified
Auto-cascade on edit No Yes (detects dirty files)
Auto-process new links Requires manual trigger Instant
Quality gates Complex config Native (blocker transformer)
File monitoring Not core feature Native (documentfeeder, etc.)
Setup time Hours to days Minutes
Iteration speed Slow (restart, retrigger) Instant (edit, cascade)

The Key Insight

CyborgShell IS the orchestrator.

You don't configure an orchestrator to run your DAG. CyborgShell actively manages the DAG:

When you set up a story generator, CyborgShell doesn't just sit there. It:

The train track: The graph stays acyclic (clean, parallelizable, understandable), but CyborgShell actively routes data through the network based on dirty files, transformers, quality gates, and conditions. It's active orchestration, not passive execution.

CyborgShell: Redefining DAG Orchestration

CyborgShell takes the DAG concept—acyclic, parallelizable, structured—and adds something missing from traditional DAG tools: integrated, reactive orchestration.

You get:

Whether it's a simple story generator or a complex AI system that breaks tasks into subtasks, monitors quality gates, and adapts based on results - CyborgShell handles the orchestration automatically.