Audio Management

The audio endpoints allow you to manage audio devices, control recording, and monitor audio pipeline health.

List Audio Devices

Get a list of all available audio input and output devices.

Endpoint

GET /audio/list

Example

curl "http://localhost:3030/audio/list"

Response

Returns an array of audio devices:

name

string

Name of the audio device

is_default

boolean

Whether this is the default device

Example Response

[
  {
    "name": "MacBook Pro Microphone",
    "is_default": true
  },
  {
    "name": "External USB Microphone",
    "is_default": false
  },
  {
    "name": "System Audio (Output)",
    "is_default": false
  }
]

Start Audio Recording

Start the audio processing pipeline to begin recording and transcribing.

Endpoint

POST /audio/start

Example

curl -X POST "http://localhost:3030/audio/start"

Response

{
  "success": true,
  "message": "Audio recording started"
}

Stop Audio Recording

Stop the audio processing pipeline.

Endpoint

POST /audio/stop

Example

curl -X POST "http://localhost:3030/audio/stop"

Response

{
  "success": true,
  "message": "Audio recording stopped"
}

Start Specific Audio Device

Start recording from a specific audio device.

Endpoint

POST /audio/device/start

Request Body

device_name

string

required

Name of the audio device to start

Example

curl -X POST "http://localhost:3030/audio/device/start" \
  -H "Content-Type: application/json" \
  -d '{
    "device_name": "External USB Microphone"
  }'

Response

success

boolean

Whether the device was started successfully

message

string

Status message

{
  "success": true,
  "message": "Device started successfully"
}

Stop Specific Audio Device

Stop recording from a specific audio device.

Endpoint

POST /audio/device/stop

Request Body

device_name

string

required

Name of the audio device to stop

Example

curl -X POST "http://localhost:3030/audio/device/stop" \
  -H "Content-Type: application/json" \
  -d '{
    "device_name": "External USB Microphone"
  }'

Response

{
  "success": true,
  "message": "Device stopped successfully"
}

Get Audio Pipeline Metrics

Get real-time metrics about the audio processing pipeline.

Endpoint

GET /audio/metrics

Example

curl "http://localhost:3030/audio/metrics"

Response

Returns detailed audio pipeline metrics:

uptime_secs

number

Pipeline uptime in seconds

chunks_sent

integer

Number of audio chunks sent for processing

chunks_channel_full

integer

Number of times the channel was full

stream_timeouts

integer

Number of stream timeout events

vad_passed

integer

Voice activity detection: chunks with speech

vad_rejected

integer

Voice activity detection: chunks without speech

vad_passthrough_rate

number

Percentage of chunks passing VAD

avg_speech_ratio

number

Average ratio of speech in audio chunks

transcriptions_completed

integer

Number of successful transcriptions

transcriptions_empty

integer

Number of empty transcriptions

transcription_errors

integer

Number of transcription errors

db_inserted

integer

Number of transcriptions inserted into database

total_words

integer

Total words transcribed

words_per_minute

number

Average words per minute

transcription_mode

string

Current transcription mode

transcription_paused

boolean

Whether transcription is paused

Example Response

{
  "uptime_secs": 3600.5,
  "chunks_sent": 1500,
  "chunks_channel_full": 2,
  "stream_timeouts": 0,
  "vad_passed": 450,
  "vad_rejected": 1050,
  "vad_passthrough_rate": 0.3,
  "avg_speech_ratio": 0.65,
  "transcriptions_completed": 420,
  "transcriptions_empty": 30,
  "transcription_errors": 5,
  "db_inserted": 415,
  "total_words": 12500,
  "words_per_minute": 208.33,
  "transcription_mode": "streaming",
  "transcription_paused": false,
  "segments_deferred": 0,
  "segments_batch_processed": 100,
  "batch_paused_reason": null
}

Speaker Management

Screenpipe automatically identifies different speakers in audio recordings. You can manage speakers through these endpoints:

Search for Audio by Speaker

Use the main search endpoint with speaker filters:

curl "http://localhost:3030/search?content_type=audio&speaker_name=john&limit=20"

Or filter by specific speaker IDs:

curl "http://localhost:3030/search?content_type=audio&speaker_ids=1,2,5&limit=20"

Use Cases

Device Setup

List available devices
Start recording from preferred device
Monitor metrics to ensure it’s working

# List devices
curl "http://localhost:3030/audio/list"

# Start specific device
curl -X POST "http://localhost:3030/audio/device/start" \
  -H "Content-Type: application/json" \
  -d '{"device_name": "External USB Microphone"}'

# Check metrics
curl "http://localhost:3030/audio/metrics"

Meeting Recording

Start audio before meeting
Stop after meeting ends
Search transcriptions with time filters

# Start recording
curl -X POST "http://localhost:3030/audio/start"

# After meeting, stop recording
curl -X POST "http://localhost:3030/audio/stop"

# Search meeting content
curl "http://localhost:3030/search?content_type=audio&start_time=2024-03-08T14:00:00Z&end_time=2024-03-08T15:00:00Z"

Multi-Device Recording

Record from both microphone and system audio simultaneously:

# Start microphone
curl -X POST "http://localhost:3030/audio/device/start" \
  -H "Content-Type: application/json" \
  -d '{"device_name": "MacBook Pro Microphone"}'

# Start system audio
curl -X POST "http://localhost:3030/audio/device/start" \
  -H "Content-Type: application/json" \
  -d '{"device_name": "System Audio (Output)"}'

Audio Pipeline Tips

Optimize Audio Recording:

Use high-quality microphones for better transcription accuracy
Monitor vad_passthrough_rate - low rates indicate mostly silence
Check transcription_errors - high counts may indicate audio issues
Review words_per_minute to verify realistic transcription rates
Use speaker filters when searching to find specific conversations

Privacy Considerations:

Audio recording captures all sound from the selected device
System audio recording captures all computer audio output
Be mindful of privacy when recording meetings or calls
Consider stopping recording during sensitive conversations
Use speaker identification to filter out unwanted recordings

Troubleshooting

No Audio Devices Found

# Check permissions
# On macOS: System Preferences > Security & Privacy > Microphone
# On Linux: Check PulseAudio/ALSA configuration
# On Windows: Check device drivers

# List devices
curl "http://localhost:3030/audio/list"

Poor Transcription Quality

Check audio metrics for high error rates
Verify microphone quality and positioning
Reduce background noise
Check VAD settings

curl "http://localhost:3030/audio/metrics"
# Look for high transcription_errors
# Check avg_speech_ratio (should be > 0.5 for good audio)

Device Already in Use

If a device fails to start:

Stop the device first
Check if another application is using it
Restart the audio pipeline

# Stop device
curl -X POST "http://localhost:3030/audio/device/stop" \
  -H "Content-Type: application/json" \
  -d '{"device_name": "MacBook Pro Microphone"}'

# Wait a moment, then start again
curl -X POST "http://localhost:3030/audio/device/start" \
  -H "Content-Type: application/json" \
  -d '{"device_name": "MacBook Pro Microphone"}'

Next Steps

Search audio transcriptions with the search endpoint
Monitor overall system health with health endpoint
Learn about pipes to process audio in real-time

Overview

Endpoints

Documentation Index

​Audio Management

​List Audio Devices

​Endpoint

​Example

​Response

​Example Response

​Start Audio Recording

​Endpoint

​Example

​Response

​Stop Audio Recording

​Endpoint

​Example

​Response

​Start Specific Audio Device

​Endpoint

​Request Body

​Example

​Response

​Stop Specific Audio Device

​Endpoint

​Request Body

​Example

​Response

​Get Audio Pipeline Metrics

​Endpoint

​Example

​Response

​Example Response

​Speaker Management

​Search for Audio by Speaker

​Use Cases

​Device Setup

​Meeting Recording

​Multi-Device Recording

​Audio Pipeline Tips

​Troubleshooting

​No Audio Devices Found

​Poor Transcription Quality

​Device Already in Use

​Next Steps

Audio Management

List Audio Devices

Endpoint

Example

Response

Example Response

Start Audio Recording

Endpoint

Example

Response

Stop Audio Recording

Endpoint

Example

Response

Start Specific Audio Device

Endpoint

Request Body

Example

Response

Stop Specific Audio Device

Endpoint

Request Body

Example

Response

Get Audio Pipeline Metrics

Endpoint

Example

Response

Example Response

Speaker Management

Search for Audio by Speaker

Use Cases

Device Setup

Meeting Recording

Multi-Device Recording

Audio Pipeline Tips

Troubleshooting

No Audio Devices Found

Poor Transcription Quality

Device Already in Use

Next Steps