Working with Media

Overview

This guide covers the complete workflow for uploading media, monitoring processing, adding chapters, and managing media assets.

Important for YouTube/Vimeo Users: Even if you’re hosting videos on YouTube or Vimeo for playback, you must still upload the original video file to Haystack. This allows Haystack to provide AI-powered features like transcription, semantic search, and chapter generation. Your embedded player will continue using YouTube/Vimeo, while Haystack provides the search and discovery layer.

Prerequisites

An item created (see Managing Content)
Media file or URL ready to upload
API token for authentication

Upload Process

Uploading media to Haystack is a two-step process:

Step 1: Create Media Asset

First, create a media asset record and receive an upload URL:

async function createMediaAsset(itemId, externalConfig = null) {
  const payload = { itemId };

  // Optional: If using YouTube/Vimeo for playback
  if (externalConfig) {
    payload.externalUrl = externalConfig.url;
    payload.externalPlatform = externalConfig.platform; // 'youtube' or 'vimeo'
  }

  const response = await fetch(`${API_URL}/media/create`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_TOKEN}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify(payload)
  });

  const { mediaAsset, uploadUrl } = await response.json();
  return { mediaAsset, uploadUrl };
}

// Example 1: Using Haystack player
const { mediaAsset, uploadUrl } = await createMediaAsset(42);

// Example 2: Using YouTube for playback
const { mediaAsset, uploadUrl } = await createMediaAsset(42, {
  url: 'https://www.youtube.com/watch?v=abc123',
  platform: 'youtube'
});

// Example 3: Using Vimeo for playback
const { mediaAsset, uploadUrl } = await createMediaAsset(42, {
  url: 'https://vimeo.com/123456789',
  platform: 'vimeo'
});

External URLs for playback only: If you provide a YouTube or Vimeo URL, it will be used for embedded playback in your application. However, you must still upload the original video file in Step 2 for AI processing.

Step 2: Upload File

Use the returned uploadUrl to upload your video or audio file:

async function uploadFile(uploadUrl, file) {
  const response = await fetch(uploadUrl, {
    method: 'PUT',
    headers: {
      'Content-Type': file.type || 'video/mp4'
    },
    body: file
  });

  if (!response.ok) {
    throw new Error('Upload failed');
  }

  return response;
}

// Usage
const videoFile = document.getElementById('fileInput').files[0];
await uploadFile(uploadUrl, videoFile);
console.log('Upload complete! Processing will begin automatically.');

File size limit: Maximum file size is 5GB. There is no way to upload files larger than 5GB.

Supported formats:

Video: MP4, MOV, AVI, MKV, WebM
Audio: MP3, WAV, AAC, M4A, FLAC

Complete Upload Function

Here’s a complete helper function that handles both steps:

async function uploadMediaToHaystack(itemId, file, externalConfig = null) {
  try {
    // Step 1: Create media asset and get upload URL
    console.log('Creating media asset...');
    const { mediaAsset, uploadUrl } = await createMediaAsset(itemId, externalConfig);

    // Step 2: Upload the file
    console.log('Uploading file...');
    await uploadFile(uploadUrl, file);

    console.log('Upload complete! Media asset ID:', mediaAsset.id);
    return mediaAsset;
  } catch (error) {
    console.error('Upload failed:', error);
    throw error;
  }
}

// Usage examples:

// With Haystack player
const videoFile = document.getElementById('fileInput').files[0];
await uploadMediaToHaystack(42, videoFile);

// With YouTube player (must still upload the file!)
await uploadMediaToHaystack(42, videoFile, {
  url: 'https://www.youtube.com/watch?v=abc123',
  platform: 'youtube'
});

Monitoring Processing Status

Polling Method

Check processing status periodically:

async function waitForProcessing(mediaAssetId, maxAttempts = 60) {
  for (let i = 0; i < maxAttempts; i++) {
    const response = await fetch(`${API_URL}/media/${mediaAssetId}`, {
      headers: {
        'Authorization': `Bearer ${API_TOKEN}`
      }
    });

    const { mediaAsset } = await response.json();

    console.log(`Status: ${mediaAsset.status}`);

    if (mediaAsset.status === 'ready') {
      console.log('Processing complete!');
      return mediaAsset;
    }

    if (mediaAsset.status === 'error') {
      throw new Error('Processing failed: ' + mediaAsset.errors);
    }

    // Wait 30 seconds before checking again
    await new Promise(resolve => setTimeout(resolve, 30000));
  }

  throw new Error('Processing timeout');
}

Avoid polling too frequently. Check every 30-60 seconds to avoid rate limits.

For the best user experience with bulk uploads, implement a background job that polls the API every 30-60 seconds to check status updates.

Adding Chapters

Automatic Chapter Detection

Request automatic chapter generation during processing:

async function processWithChapters(itemId) {
  const response = await fetch(`${API_URL}/items/${itemId}/process`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_TOKEN}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      generateChapters: true
    })
  });

  const { item } = await response.json();
  return item;
}

Manual Chapters

Create custom chapters with precise control:

async function createChapter(itemId, mediaAssetId, title, startMs) {
  const response = await fetch(`${API_URL}/media/chapters`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_TOKEN}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      itemId,
      mediaAssetId,
      title,
      startMs,
      suggested: false,
      accepted: true
    })
  });

  const { chapter } = await response.json();
  return chapter;
}

// Example: Create multiple chapters
const chapters = [
  { title: 'Welcome & Announcements', startMs: 0 },
  { title: 'Worship', startMs: 180000 },
  { title: 'Message: The Power of Prayer', startMs: 1200000 },
  { title: 'Altar Call', startMs: 2400000 },
  { title: 'Closing Prayer', startMs: 2700000 }
];

for (const chapterData of chapters) {
  await createChapter(42, 100, chapterData.title, chapterData.startMs);
}

Chapter best practices:

Use 5-15 minute segments
Start each chapter at a natural transition
Use descriptive, clear titles
Order chapters chronologically (handled automatically by sortOrder)

Managing Thumbnails

Thumbnail availability depends on which video player you’re using.

For Haystack Custom Player

When using the Haystack custom player, thumbnails are automatically generated via Mux. Access them via the Mux image API:

function getThumbnailUrl(muxPlaybackId, options = {}) {
  const {
    time = 0,        // Seconds into video
    width = 640,     // Width in pixels
    height = null,   // Auto if not specified
    fitMode = 'smartcrop'
  } = options;

  const params = new URLSearchParams({
    time: time.toString(),
    width: width.toString(),
    fit_mode: fitMode
  });

  if (height) params.set('height', height.toString());

  return `https://image.mux.com/${muxPlaybackId}/thumbnail.jpg?${params}`;
}

// Usage
const url = getThumbnailUrl('abc123def456', {
  time: 120,  // 2 minutes in
  width: 1280
});

Extract thumbnails from key moments:

// Get thumbnails for each chapter
chapters.forEach(chapter => {
  const thumbnailUrl = getThumbnailUrl(muxPlaybackId, {
    time: chapter.startTimeSeconds,
    width: 640
  });

  console.log(`${chapter.title}: ${thumbnailUrl}`);
});

For YouTube/Vimeo Players

Mux-generated thumbnails are not available when using YouTube or Vimeo for playback. Use the artwork files you uploaded to your items instead.

When using YouTube or Vimeo players, retrieve thumbnails from your item’s artwork:

async function getItemArtwork(itemId) {
  const response = await fetch(`${API_URL}/items/${itemId}`, {
    headers: {
      'Authorization': `Bearer ${API_TOKEN}`
    }
  });

  const { item } = await response.json();

  // Artwork URLs are directly on the item
  return {
    square: item.squareImgUrl,
    wide: item.wideImgUrl,
    ultraWide: item.ultraWideImgUrl,
    vertical: item.verticalImgUrl
  };
}

// Usage
const artwork = await getItemArtwork(42);
console.log('Square thumbnail:', artwork.square);
console.log('Wide thumbnail:', artwork.wide);
console.log('Ultra-wide thumbnail:', artwork.ultraWide);
console.log('Vertical thumbnail:', artwork.vertical);

Playback Integration

HLS Streaming

Use the HLS playback URL with any HLS-compatible player:

async function getPlaybackUrl(mediaAssetId) {
  const response = await fetch(`${API_URL}/media/${mediaAssetId}`, {
    headers: {
      'Authorization': `Bearer ${API_TOKEN}`
    }
  });

  const { mediaAsset } = await response.json();
  return `https://stream.mux.com/${mediaAsset.muxPlaybackId}.m3u8`;
}

// Usage with Video.js
const playbackUrl = await getPlaybackUrl(100);

const player = videojs('my-video', {
  sources: [{
    src: playbackUrl,
    type: 'application/x-mpegURL'
  }]
});

HLS.js Example

<video id="video" controls></video>

<script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
<script>
  const video = document.getElementById('video');
  const playbackUrl = 'https://stream.mux.com/abc123.m3u8';

  if (Hls.isSupported()) {
    const hls = new Hls();
    hls.loadSource(playbackUrl);
    hls.attachMedia(video);
  } else if (video.canPlayType('application/vnd.apple.mpegurl')) {
    // Native HLS support (Safari)
    video.src = playbackUrl;
  }
</script>

Multiple Media Assets

Items can have multiple media assets (e.g., audio and video versions):

// Upload video version
const videoAsset = await uploadMedia(itemId, videoFile);

// Upload audio-only version
const audioAsset = await uploadMedia(itemId, audioFile);

// List all media for an item
const response = await fetch(
  `${API_URL}/media?itemId=${itemId}`,
  {
    headers: {
      'Authorization': `Bearer ${API_TOKEN}`
    }
  }
);

const { mediaAssets } = await response.json();
console.log(`Item has ${mediaAssets.length} media assets`);

Updating Media

Replace Media

To replace media, delete the old asset and upload a new one:

// Delete old media
await fetch(`${API_URL}/media/${oldMediaAssetId}`, {
  method: 'DELETE',
  headers: {
    'Authorization': `Bearer ${API_TOKEN}`
  }
});

// Upload new media
const newAsset = await uploadMedia(itemId, newFile);

Deleting media is permanent and cannot be undone. Ensure you have backups if needed.

Complete Upload Workflow

Here’s a complete example combining all steps:

async function completeMediaWorkflow(itemId, videoFile, externalConfig = null) {
  try {
    // 1. Upload media
    console.log('Uploading media...');
    const mediaAsset = await uploadMediaToHaystack(itemId, videoFile, externalConfig);

    // 2. Wait for processing
    console.log('Processing media...');
    const processedAsset = await waitForProcessing(mediaAsset.id);

    // 3. Add chapters
    console.log('Adding chapters...');
    const chapters = [
      { title: 'Introduction', startTimeSeconds: 0 },
      { title: 'Main Content', startTimeSeconds: 300 },
      { title: 'Conclusion', startTimeSeconds: 2400 }
    ];
    await createChapters(processedAsset.id, chapters);

    // 4. Publish the item
    console.log('Publishing item...');
    await fetch(`${API_URL}/items/${itemId}/publish`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${API_TOKEN}`
      }
    });

    console.log('✓ Complete! Item is now live.');
    return processedAsset;

  } catch (error) {
    console.error('Workflow failed:', error.message);
    throw error;
  }
}

Troubleshooting

Common Issues

Upload Fails or Times Out

Causes:

File too large (>5GB for direct upload)
Poor internet connection
Unsupported codec

Solutions:

Use external URL method for large files
Compress video before upload
Ensure video uses H.264 codec

Processing Fails

Causes:

Corrupt video file
Invalid URL
Unsupported format

Solutions:

Test playback locally before upload
Ensure URL is publicly accessible
Convert to MP4 with H.264/AAC

Transcription Quality Poor

Causes:

Low audio quality
Background noise
Multiple overlapping speakers

Solutions:

Use high-quality audio recording
Remove background noise in post-production
Ensure clear, single speaker audio

Slow Processing

Causes:

Very long videos (>2 hours)
4K resolution
High platform load

Solutions:

Expect 1-2x realtime processing
Poll every 30-60 seconds to check status
Consider splitting very large files into segments

Best Practices

Optimize Before Upload

Compress videos to reasonable bitrates (5-10 Mbps for 1080p) to save storage and bandwidth

Poll Responsibly

Check processing status every 30-60 seconds to avoid rate limits

Add Chapters

Break long content into chapters for better user experience and engagement

Test Playback

Verify video plays correctly before uploading to catch issues early

Next Steps

Media Management Concepts

Deep dive into media processing pipeline

Getting Started

Core Concepts

Guides

Overview

Prerequisites

Upload Process

Step 1: Create Media Asset

Step 2: Upload File

Complete Upload Function

Monitoring Processing Status

Polling Method

Adding Chapters

Automatic Chapter Detection

Manual Chapters

Managing Thumbnails

For Haystack Custom Player

For YouTube/Vimeo Players

Playback Integration

HLS Streaming

HLS.js Example

Multiple Media Assets

Updating Media

Replace Media

Complete Upload Workflow

Troubleshooting

Common Issues

Best Practices

Optimize Before Upload

Poll Responsibly

Add Chapters

Test Playback

Next Steps

Media Management Concepts

Getting Started

Core Concepts

Guides

​Overview

​Prerequisites

​Upload Process

​Step 1: Create Media Asset

​Step 2: Upload File

​Complete Upload Function

​Monitoring Processing Status

​Polling Method

​Adding Chapters

​Automatic Chapter Detection

​Manual Chapters

​Managing Thumbnails

​For Haystack Custom Player

​For YouTube/Vimeo Players

​Playback Integration

​HLS Streaming

​HLS.js Example

​Multiple Media Assets

​Updating Media

​Replace Media

​Complete Upload Workflow

​Troubleshooting

​Common Issues

​Best Practices

Optimize Before Upload

Poll Responsibly

Add Chapters

Test Playback

​Next Steps

Media Management Concepts

Overview

Prerequisites

Upload Process

Step 1: Create Media Asset

Step 2: Upload File

Complete Upload Function

Monitoring Processing Status

Polling Method

Adding Chapters

Automatic Chapter Detection

Manual Chapters

Managing Thumbnails

For Haystack Custom Player

For YouTube/Vimeo Players

Playback Integration

HLS Streaming

HLS.js Example

Multiple Media Assets

Updating Media

Replace Media

Complete Upload Workflow

Troubleshooting

Common Issues

Best Practices

Next Steps