> ## Documentation Index
> Fetch the complete documentation index at: https://developer.thehaystack.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Media Management

> How Haystack handles video and audio assets

## Overview

Haystack provides comprehensive media management capabilities including automatic transcription, multi-format encoding, chapter detection, and CDN delivery. Understanding the media pipeline helps you efficiently manage your content library.

## Video Hosting Options

Churches can choose how they want to host and play their video content:

<CardGroup cols={3}>
  <Card title="Haystack Player" icon="play">
    Use Haystack's custom video player powered by Mux
  </Card>

  <Card title="YouTube" icon="youtube">
    Embed YouTube player in your site
  </Card>

  <Card title="Vimeo" icon="vimeo-v">
    Embed Vimeo player in your site
  </Card>
</CardGroup>

<Warning>
  **Important:** Regardless of which player you choose, you must upload the original video or audio file to Haystack for AI processing. Even if you host videos on YouTube or Vimeo, Haystack needs the source file for transcription, search indexing, and other AI features.
</Warning>

### Why Upload to Haystack?

When you upload media to Haystack (even if hosting elsewhere), we:

* **Transcribe** the audio with word-level timestamps
* **Index** the content for semantic search
* **Extract** topics, speakers, and key moments
* **Generate** searchable chapters
* **Enable** "jump to moment" functionality

The original file stays on YouTube/Vimeo for playback, while Haystack provides the AI-powered search and discovery features.

## Media Assets

**Media Assets** represent video or audio files attached to items. Each item can have multiple media assets in different formats or qualities.

```json theme={null}
{
  "id": 100,
  "itemId": 42,
  "contentType": "video",
  "status": "ready",
  "duration": 2847.5,
  "muxAssetId": "abc123",
  "muxPlaybackId": "def456"
}
```

### Content Types

Haystack supports two primary content types:

<CardGroup cols={2}>
  <Card title="Video" icon="video">
    Full video content with audio, visual, and subtitle tracks
  </Card>

  <Card title="Audio" icon="podcast">
    Audio-only content for podcasts and audio messages
  </Card>
</CardGroup>

### Media Status Lifecycle

<Steps>
  <Step title="created">
    Media asset record created, awaiting file upload
  </Step>

  <Step title="uploaded">
    File uploaded, awaiting processing
  </Step>

  <Step title="processing">
    Transcoding to multiple formats and bitrates
  </Step>

  <Step title="ready">
    Available for playback and searching
  </Step>

  <Step title="error">
    Processing failed - check error details
  </Step>
</Steps>

## Creating Media Assets

To upload a video or audio file to Haystack, use a two-step process:

### Step 1: Create Media Asset and Get Upload URL

First, create a media asset to get a secure upload URL:

```javascript theme={null}
const response = await fetch(`${API_URL}/media/create`, {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${API_TOKEN}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    itemId: 42,
    // Optional: If using YouTube/Vimeo for playback
    externalUrl: 'https://www.youtube.com/watch?v=abc123',
    externalPlatform: 'youtube' // or 'vimeo'
  })
});

const { mediaAsset, uploadUrl } = await response.json();
console.log('Upload URL:', uploadUrl);
```

**Parameters:**

* `itemId` (required) - The item this media belongs to
* `externalUrl` (optional) - YouTube or Vimeo URL for playback only
* `externalPlatform` (optional) - `'youtube'` or `'vimeo'` when using external playback

<Note>
  **External URLs are for playback only.** Even if you provide a YouTube or Vimeo URL, you must still upload the original video file to Haystack in Step 2 for AI processing (transcription, search, etc.).
</Note>

### Step 2: Upload File to Upload URL

Use the returned `uploadUrl` to upload your video or audio file:

```javascript theme={null}
// Upload the file using the upload URL
const uploadResponse = await fetch(uploadUrl, {
  method: 'PUT',
  headers: {
    'Content-Type': 'video/mp4', // or appropriate content type
  },
  body: videoFile // File object or buffer
});

if (uploadResponse.ok) {
  console.log('Upload complete!');
  console.log('Media asset ID:', mediaAsset.id);
}
```

<Warning>
  **File size limit:** 5GB maximum. Files larger than 5GB cannot be uploaded.
</Warning>

**Supported formats:**

* Video: MP4, MOV, AVI, MKV, WebM
* Audio: MP3, WAV, AAC, M4A, FLAC

## Automatic Transcription

When media is uploaded, Haystack automatically:

1. **Extracts audio** from video files
2. **Transcribes speech** using AI speech recognition
3. **Timestamps every word** for precise searching
4. **Indexes content** for semantic search

### Transcription Quality

Transcription accuracy depends on:

* Audio quality (higher quality = better accuracy)
* Speaker clarity and accent
* Background noise levels
* Language (English is best supported)

<Tip>
  For best transcription results, use high-quality audio recordings with minimal background noise.
</Tip>

## Media Chapters

**Chapters** divide media into logical sections with titles and timestamps. They improve navigation and user experience.

```json theme={null}
{
  "id": 50,
  "mediaAssetId": 100,
  "title": "Introduction",
  "startTimeSeconds": 0,
  "sortOrder": 1
}
```

### Auto-Generated Chapters

Haystack can automatically detect chapter boundaries based on:

* Transcript analysis
* Silence detection
* Scene changes (for video)

### Manual Chapters

Create chapters manually for precise control:

```javascript theme={null}
const chapters = [
  { title: "Introduction", startMs: 0 },
  { title: "Main Teaching", startMs: 180000 },
  { title: "Q&A", startMs: 1200000 },
  { title: "Closing Prayer", startMs: 2400000 }
];

// Create chapters individually
for (const chapter of chapters) {
  await fetch(`${API_URL}/media/chapters`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_TOKEN}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      itemId: 42,
      mediaAssetId: 100,
      title: chapter.title,
      startMs: chapter.startMs,
      suggested: false,
      accepted: true
    })
  });
}
```

## Media Variants

**Variants** allow you to offer different versions of content for the same item. Each collection has predefined variant types that serve different purposes.

### Common Variant Types

* **Sermon Video** - The main sermon message (video)
* **Sermon Audio** - Audio-only version of the sermon
* **Full Service** - Complete worship service including music, announcements, etc. (video)
* **Podcast** - Edited audio version optimized for podcast distribution

### How Variants Work

Each item can have multiple media assets, each assigned to a different variant type:

```json theme={null}
{
  "item": {
    "id": 42,
    "title": "Leading with Humility",
    "mediaAssets": [
      {
        "id": 100,
        "variantType": {
          "id": 1,
          "name": "Sermon Video",
          "contentType": "video",
          "indexable": true
        }
      },
      {
        "id": 101,
        "variantType": {
          "id": 2,
          "name": "Sermon Audio",
          "contentType": "audio",
          "indexable": true
        }
      },
      {
        "id": 102,
        "variantType": {
          "id": 3,
          "name": "Full Service",
          "contentType": "video",
          "indexable": false
        }
      }
    ]
  }
}
```

### Variant Properties

* **name** - Descriptive name (e.g., "Sermon Video", "Podcast")
* **contentType** - Either "video" or "audio"
* **indexable** - Whether this variant can be used for search/transcription
* **displayOrder** - Order shown to users when selecting variants
* **collectionId** - Variant types are tied to specific collections

### Indexing and Search

Only **one** media asset per item is indexed for search and transcription:

* If multiple variants are marked as `indexable`, video variants are preferred over audio
* The indexed variant is used for transcription, search, and AI features
* All other variants are available for playback but not searchable

<Note>
  Variant types are predefined per collection and cannot be customized by API consumers. Each collection includes video and audio variant types by default.
</Note>

## Playback URLs

Haystack stores the Mux playback ID, which you use to construct playback URLs:

```json theme={null}
{
  "mediaAsset": {
    "id": 100,
    "muxPlaybackId": "abc123def456"
  }
}
```

Construct the HLS playback URL using the `muxPlaybackId`:

```javascript theme={null}
const playbackUrl = `https://stream.mux.com/${mediaAsset.muxPlaybackId}.m3u8`;
```

### HLS Streaming

All media is delivered via **HLS (HTTP Live Streaming)** for:

* Adaptive bitrate streaming
* Fast startup times
* Reliable playback across devices

### Thumbnails

**For Haystack Custom Player:**

When using the Haystack custom player, thumbnails are automatically generated at regular intervals via Mux:

```javascript theme={null}
// Get thumbnail at 30 seconds
const thumbnailUrl = `https://image.mux.com/${muxPlaybackId}/thumbnail.jpg?time=30`;

// Get thumbnail at specific width
const thumbnail = `https://image.mux.com/${muxPlaybackId}/thumbnail.jpg?width=640&time=120`;
```

**For YouTube/Vimeo Players:**

When using YouTube or Vimeo for playback, Mux thumbnails are not available. Use the artwork files you uploaded to your items instead.

<Note>
  Thumbnail availability depends on your chosen video player. Mux-generated thumbnails are only available when using the Haystack custom player.
</Note>

## API Reference

Key endpoints for media management:

* `POST /media/create` - Create media asset and get upload URL
* `GET /media/{id}` - Get media asset details
* `DELETE /media/{id}` - Delete media asset
* `POST /media/chapters` - Create a chapter
* `PATCH /media/chapters/{id}` - Update a chapter
* `DELETE /media/chapters/{id}` - Delete a chapter

## Next Steps

<CardGroup cols={2}>
  <Card title="Search & Discovery" icon="magnifying-glass" href="/concepts/search-discovery">
    Learn how transcribed content powers semantic search
  </Card>

  <Card title="Working with Media Guide" icon="video" href="/guides/working-with-media">
    Complete guide to uploading and managing media
  </Card>
</CardGroup>