Skip to content
11 changes: 6 additions & 5 deletions .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "runway-api",
"description": "Helps users integrate Runway's public API (https://docs.dev.runwayml.com/) into their projects. Analyzes codebase compatibility, guides API key setup, and provides hands-on assistance for video generation, image generation, audio, file uploads, and direct API actions.",
"version": "1.2.0",
"description": "Video generation at scale. Generate videos, images, and audio with Runway's API — batch ad campaigns, product videos, multishot stories, and creative iteration. Supports seedance2, gen4.5, veo3, Nano, Banana Pro, and more.",
"version": "2.0.0",
"author": {
"name": "Runway",
"email": "ops@runwayml.com"
Expand All @@ -11,12 +11,13 @@
"runwayml",
"video-generation",
"image-generation",
"audio-generation",
"ai-api",
"seedance2",
"gen4",
"veo3",
"characters",
"avatars",
"public-api",
"media-generation",
"batch",
"runway-api"
]
}
17 changes: 6 additions & 11 deletions .cursor-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"name": "runway-api",
"displayName": "Runway API",
"version": "1.0.1",
"description": "Helps users integrate Runway's public API (https://docs.dev.runwayml.com/) into their projects. Analyzes codebase compatibility, guides API key setup, and provides hands-on assistance for video generation, image generation, audio, file uploads, and direct API actions.",
"version": "2.0.0",
"description": "Video generation at scale. Generate videos, images, and audio with Runway's API — batch ad campaigns, product videos, multishot stories, and creative iteration. Supports seedance2, gen4.5, veo3, Nano, Banana Pro, and more.",
"author": {
"name": "Runway",
"email": "ops@runwayml.com"
Expand All @@ -18,18 +18,13 @@
"image-generation",
"audio-generation",
"ai-api",
"seedance2",
"gen4",
"veo3",
"characters",
"avatars"
],
"category": "ai-tools",
"tags": [
"api-integration",
"media-generation",
"video",
"image",
"audio"
"batch"
],
"category": "ai-tools",
"tags": ["media-generation", "video", "image", "audio", "api-integration"],
"skills": "./skills/"
}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@ __pycache__/

# Dependencies
node_modules/

# Test output files
test-*
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Changelog

## 2.0.0

- **Media generation:** New `rw-generate-video`, `rw-generate-image`, `rw-generate-audio` skills that run Python scripts directly via `uv run` — no SDK setup required
- **Runnable scripts:** Added `scripts/` directory with `generate_video.py`, `generate_image.py`, `generate_audio.py`, `list_models.py`, `get_task.py`, and shared `runway_helpers.py`
- **Seedance 2 support:** Added `seedance2` model across all generation scripts and skills (TTV, ITV, VTV, 36 credits/sec)
- **Plugin metadata:** Updated descriptions and keywords for both Claude and Cursor plugins

## 1.1.0

- **Breaking change:** Every skill is now named with an `rw-` prefix (skill folder under `skills/`, `name` in each `SKILL.md`, and `+…` invocations). Examples: `setup-api-key` → `rw-setup-api-key`, `integrate-video` → `rw-integrate-video`. Update any documentation, shortcuts, or automation that referenced the previous names or paths.
Expand Down
93 changes: 48 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,39 @@
# Runway API Skills

A set of skills that gives your AI coding agent the knowledge and tools to work with [Runway's public API](https://docs.dev.runwayml.com/api/)integrate video, image, and audio generation into server-side projects, or manage Runway resources directly from the editor.
Video generation at scale. Generate videos, images, and audio with Runway's API — batch Ad campaigns, product videos, multishot stories, and creative iteration. Supports seedance2, gen4.5, veo3, Nano, Banana Pro, and more.

Works with [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://openai.com/index/codex/), and other compatible agents.
Works with [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Cursor](https://cursor.com), [Codex](https://openai.com/index/codex/), and other compatible agents.

## Two Ways to Use

### 1. Integrate into your app
### 1. Generate media at scale

Guide your agent through adding Runway capabilities to a server-side project: verify compatibility, set up credentials, write framework-specific routes, and handle edge cases like file uploads and task polling.
Tell your agent what to create and it handles the rest — calls the API, polls for completion, downloads the result. Generate one asset or orchestrate hundreds.

```
Set up Runway video generation in my Next.js app
Generate a 15-second product video from this image using seedance2
```

### 2. Act on your Runway account directly

Let your agent call the Runway API to manage resources — create avatars, update knowledge documents, trigger generations, check credit balance — without writing app code.
```
Generate a video for each of these 5 product photos, 9:16 ratio for Instagram Reels
```

```
Create a new avatar called "Support Agent" with a friendly personality
Create a voiceover for this ad script, then generate a sound effect of applause
```

### 2. Integrate into your app

Guide your agent through adding Runway capabilities to a server-side project: verify compatibility, set up credentials, write framework-specific routes, and handle edge cases like file uploads and task polling.

```
List my knowledge documents for avatar abc123
Set up Runway video generation in my Next.js app
```

## Installation

### Claude Code (community marketplace)

Add the [Anthropic community plugins](https://github.com/anthropics/claude-plugins-community) marketplace, then install this plugin.

```bash
claude plugin marketplace add anthropics/claude-plugins-community
claude plugin install runway-api-skills@claude-community
Expand All @@ -50,34 +52,40 @@ Select all the skills with your keyboard (Space to select, arrow keys to navigat
## Prerequisites

- A [Runway developer account](https://dev.runwayml.com/) with prepaid credits ($10 minimum)
- For generation skills: [uv](https://docs.astral.sh/uv/) (Python package runner) and `RUNWAYML_API_SECRET` env var
- For integration skills: a server-side project — Node.js 18+ or Python 3.8+ with a backend framework
- For direct API actions: Node.js 20+ (zero dependencies — just the runtime script)

## Available Skills

### Getting Started
### Generation (run directly)

| Skill | Description |
| ------------------------ | ---------------------------------------------------------------------------------------------- |
| `rw-recipe-full-setup` | End-to-end setup: compatibility check → API key → SDK install → integration code → test |
| `rw-check-compatibility` | Analyze your project to verify it can safely call the Runway API server-side |
| `rw-setup-api-key` | Guide through account creation, SDK installation, and environment variable configuration |
| `rw-check-org-details` | Query your organization's rate limits, credit balance, usage tier, and daily generation counts |
Generate media assets directly — your agent runs the scripts, polls for completion, and saves the output.

### Direct API Actions
| Skill | Description |
| -------------------- | ----------------------------------------------------------------------------------- |
| `rw-generate-video` | Generate videos: text-to-video, image-to-video, video-to-video |
| `rw-generate-image` | Generate images: text-to-image with optional reference images |
| `rw-generate-audio` | Generate audio: TTS, sound effects, voice isolation, dubbing, voice conversion |

| Skill | Description |
| -------------------- | ---------------------------------------------------------------------------------- |
| `use-runway-api` | Call any public API endpoint to manage resources, trigger generations, and inspect state |
### Integration (add to your app)

### Generation (Integration)
Add Runway generation to your server-side project with framework-specific code.

| Skill | Description |
| -------------------- | ----------------------------------------------------------------------------------- |
| `rw-integrate-video` | Text-to-video, image-to-video, video-to-video, and character performance generation |
| `rw-integrate-image` | Text-to-image generation with optional reference images via `@Tag` syntax |
| `rw-integrate-audio` | Text-to-speech, sound effects, voice isolation, dubbing, and speech-to-speech |

### Getting Started

| Skill | Description |
| ------------------------ | ---------------------------------------------------------------------------------------------- |
| `rw-recipe-full-setup` | End-to-end setup: compatibility check → API key → SDK install → integration code → test |
| `rw-check-compatibility` | Analyze your project to verify it can safely call the Runway API server-side |
| `rw-setup-api-key` | Guide through account creation, SDK installation, and environment variable configuration |
| `rw-check-org-details` | Query your organization's rate limits, credit balance, usage tier, and daily generation counts |

### Characters (Real-Time Avatars)

| Skill | Description |
Expand All @@ -90,6 +98,7 @@ Select all the skills with your keyboard (Space to select, arrow keys to navigat

| Skill | Description |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------- |
| `use-runway-api` | Call any public API endpoint to manage resources, trigger generations, and inspect state |
| `rw-integrate-uploads` | Upload local files to get `runway://` URIs for use as generation inputs |
| `rw-api-reference` | Complete API reference — models, endpoints, costs, rate limits, and error codes |
| `rw-fetch-api-reference` | Fetch the latest API docs from [docs.dev.runwayml.com/api](https://docs.dev.runwayml.com/api/) as the source of truth |
Expand All @@ -98,14 +107,14 @@ Select all the skills with your keyboard (Space to select, arrow keys to navigat

### Video

| Model | Use Case | Cost |
| ------------------------ | ----------------------------------- | ----------------- |
| Model | Use Case | Cost |
| ------------------------ | ---------------------------------------- | ----------------- |
| `seedance2` | Reference image and video, long duration | 36 credits/sec |
| `gen4.5` | High quality, general purpose | 12 credits/sec |
| `gen4_turbo` | Fast, image-driven (image required) | 5 credits/sec |
| `gen4_aleph` | Video-to-video editing | 15 credits/sec |
| `veo3` | Premium quality | 40 credits/sec |
| `veo3.1` / `veo3.1_fast` | High quality / fast Google models | 10–40 credits/sec |
| `gen4.5` | High quality, general purpose | 12 credits/sec |
| `gen4_turbo` | Fast, image-driven (image required) | 5 credits/sec |
| `gen4_aleph` | Video-to-video editing | 15 credits/sec |
| `veo3` | Premium quality | 40 credits/sec |
| `veo3.1` / `veo3.1_fast` | High quality / fast Google models | 10–40 credits/sec |

### Image

Expand Down Expand Up @@ -133,34 +142,28 @@ Select all the skills with your keyboard (Space to select, arrow keys to navigat

## Quick Start

### Integrate into a project
### Generate media directly

```
Set up Runway video generation in my Next.js app
Generate a 10-second video of a sunset over the ocean
```

Or go step by step:

```
Check if my project is compatible with the Runway API
Generate an image of a red door in a white wall
```

```
Help me set up my Runway API key
Create a voiceover saying "Welcome to our store" and save it as welcome.mp3
```

```
Add an endpoint to generate videos from text prompts
```

### Direct API actions
### Integrate into a project

```
List all my avatars
Set up Runway video generation in my Next.js app
```

```
Using the Runway API, generate an image of a red door in a white wall and tell me which model you used
Add an endpoint to generate videos from text prompts
```

## Supported Frameworks
Expand Down
131 changes: 131 additions & 0 deletions scripts/generate_audio.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
# /// script
# requires-python = ">=3.10"
# dependencies = ["requests"]
# ///

"""Generate audio using the Runway API (TTS, sound effects, voice isolation, dubbing)."""

import argparse
import os
import sys

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from runway_helpers import (
get_api_key,
api_post,
poll_task,
download_file,
ensure_url,
output_path,
)

AUDIO_TYPES = {
"tts": {
"endpoint": "/v1/text_to_speech",
"model": "eleven_multilingual_v2",
"description": "Text to speech",
},
"sfx": {
"endpoint": "/v1/sound_effect",
"model": "eleven_text_to_sound_v2",
"description": "Sound effect generation",
},
"isolate": {
"endpoint": "/v1/voice_isolation",
"model": "eleven_voice_isolation",
"description": "Isolate voice from audio",
},
"dub": {
"endpoint": "/v1/voice_dubbing",
"model": "eleven_voice_dubbing",
"description": "Dub to another language",
},
"sts": {
"endpoint": "/v1/speech_to_speech",
"model": "eleven_multilingual_sts_v2",
"description": "Voice conversion",
},
}


def main():
parser = argparse.ArgumentParser(description="Generate audio with the Runway API")
parser.add_argument("--filename", required=True, help="Output filename (e.g. output.mp3)")
parser.add_argument(
"--type",
required=True,
choices=list(AUDIO_TYPES.keys()),
help="Audio type: tts, sfx, isolate, dub, sts",
)
parser.add_argument("--text", help="Text input (required for tts and sfx)")
parser.add_argument("--audio-url", help="Audio URL or local path (for isolate, dub, sts)")
parser.add_argument("--voice-id", help="Voice ID (for tts and sts)")
parser.add_argument("--target-language", help="Target language code (for dub, e.g. 'es')")
parser.add_argument("--output-dir", help="Output directory (default: cwd)")
parser.add_argument("--api-key", help="Runway API key (or set RUNWAYML_API_SECRET)")
args = parser.parse_args()

api_key = get_api_key(args.api_key)
audio_type = AUDIO_TYPES[args.type]
endpoint = audio_type["endpoint"]
model = audio_type["model"]

body = {"model": model}

if args.type == "tts":
if not args.text:
print("Error: --text is required for tts.", file=sys.stderr)
sys.exit(1)
body["promptText"] = args.text
body["voice"] = {"type": "runway-preset", "presetId": args.voice_id or "Maya"}

elif args.type == "sfx":
if not args.text:
print("Error: --text is required for sfx.", file=sys.stderr)
sys.exit(1)
body["promptText"] = args.text

elif args.type == "isolate":
if not args.audio_url:
print("Error: --audio-url is required for isolate.", file=sys.stderr)
sys.exit(1)
body["audioUri"] = ensure_url(args.audio_url, api_key)

elif args.type == "sts":
if not args.audio_url:
print("Error: --audio-url is required for sts.", file=sys.stderr)
sys.exit(1)
audio_uri = ensure_url(args.audio_url, api_key)
body["media"] = {"type": "audio", "uri": audio_uri}
body["voice"] = {"type": "runway-preset", "presetId": args.voice_id or "Maya"}

elif args.type == "dub":
if not args.audio_url:
print("Error: --audio-url is required for dub.", file=sys.stderr)
sys.exit(1)
if not args.target_language:
print("Error: --target-language is required for dub.", file=sys.stderr)
sys.exit(1)
body["audioUri"] = ensure_url(args.audio_url, api_key)
body["targetLang"] = args.target_language

print(f"Generating audio ({args.type}) with {model}...", file=sys.stderr)
task = api_post(api_key, endpoint, body)
task_id = task.get("id")
print(f"Task created: {task_id}", file=sys.stderr)

result = poll_task(api_key, task_id)
urls = result.get("output", [])

if not urls:
print("Error: No output URLs in result.", file=sys.stderr)
sys.exit(1)

out = output_path(args.filename, args.output_dir)
path = download_file(urls[0], out)
print(path)
print(f"Saved: {path}", file=sys.stderr)


if __name__ == "__main__":
main()
Loading