Search

Search screen and audio content with filters

Search Content

Query Screenpipe for content based on various filters. The search endpoint is the primary way to retrieve contextual data from your screen recordings and audio transcriptions.

Endpoint

GET /search

Use Cases

• Search for specific text across all applications
• Find content from a specific application or window
• Get screenshots from a particular time period
• Retrieve all visits to a specific website

Query Parameters

q · string

Search query string. Searches across OCR text, audio transcriptions, and UI elements.

Example: "authentication", "meeting notes"

limit · integer · default: "20"

Maximum number of results to return per page.

Range: 1-1000

offset · integer · default: "0"

Number of results to skip for pagination.

Use with limit for paginated results.

content_type · string · default: "all"

Type of content to search for.

Options:

• all - All content types
• ocr - Only screen text (OCR)
• audio - Only audio transcriptions
• ui - Only UI element text
• audio+ui - Audio and UI content
• ocr+ui - OCR and UI content
• audio+ocr - Audio and OCR content

start_time · string

Filter results after this timestamp.

Format: ISO 8601 date-time (e.g., 2024-03-08T10:00:00Z)

end_time · string

Filter results before this timestamp.

Format: ISO 8601 date-time (e.g., 2024-03-08T18:00:00Z)

app_name · string

Filter by application name.

Example: "Chrome", "Slack", "VS Code"

window_name · string

Filter by window title.

Example: "GitHub - Pull Requests"

browser_url · string

Filter by browser URL (requires browser extension or accessibility access).

Example: "github.com", "docs.screenpi.pe"

include_frames · boolean · default: "false"

Include base64-encoded frame images in the response.

Setting this to true significantly increases response size.

frame_name · string

Filter by frame name/identifier.

min_length · integer

Minimum length of text content in characters.

max_length · integer

Maximum length of text content in characters.

speaker_ids · string

Comma-separated list of speaker IDs to filter audio content.

Example: "1,2,5"

speaker_name · string

Filter audio transcriptions by speaker name (case-insensitive partial match).

Example: "john"

focused · boolean

Filter by whether the window was focused.

• true - Only focused windows
• false - Only unfocused windows

include_cloud · boolean · default: "false"

Include cloud-synced data in search results.

Requires cloud sync to be enabled.

Examples

Basic Search

Search for recent content containing “authentication”:

curl "http://localhost:3030/search?q=authentication&limit=20"

Search by Application

Find all content from Chrome in the last 7 days:

curl "http://localhost:3030/search?app_name=Chrome&start_time=2024-03-01T00:00:00Z&limit=50"

Search Browser Activity

Get all GitHub activity with screenshots:

curl "http://localhost:3030/search?browser_url=github.com&content_type=ocr&include_frames=true&limit=20"

Search Audio Transcriptions

Find meeting transcriptions from a specific speaker:

curl "http://localhost:3030/search?content_type=audio&speaker_name=john&start_time=2024-03-08T09:00:00Z&end_time=2024-03-08T17:00:00Z"

Time Range Search

Get all activity during work hours:

curl "http://localhost:3030/search?start_time=2024-03-08T09:00:00Z&end_time=2024-03-08T17:00:00Z&limit=100"

Response

data · array

Array of content items matching the search criteria.

Show ContentItem

type · string

Type of content: OCR, Audio, UI, or Input

content · object

Content data (structure varies by type)

pagination · object

Pagination information

Show properties

limit · integer

Number of results per page

offset · integer

Current offset

total · integer

Total number of matching results

cloud · object

Cloud search metadata (only present when cloud sync is available)

Show properties

cloud_search_available · boolean

Whether cloud search is available

estimated_cloud_results · integer

Estimated number of results in cloud

OCR Content

{
  "type": "OCR",
  "content": {
    "frame_id": 12345,
    "text": "import React from 'react'",
    "timestamp": "2024-03-08T14:30:00Z",
    "file_path": "/path/to/frame.mp4",
    "offset_index": 150,
    "app_name": "VS Code",
    "window_name": "main.tsx - myproject",
    "tags": ["code", "typescript"],
    "frame": null,
    "browser_url": null,
    "focused": true,
    "device_name": "MacBook Pro"
  }
}

Audio Content

{
  "type": "Audio",
  "content": {
    "chunk_id": 789,
    "transcription": "Let's discuss the authentication implementation",
    "timestamp": "2024-03-08T14:30:00Z",
    "file_path": "/path/to/audio.mp4",
    "offset_index": 45,
    "tags": [],
    "device_name": "MacBook Microphone",
    "device_type": "Input",
    "speaker": {
      "id": 1,
      "name": "John Doe",
      "metadata": "{}"
    },
    "start_time": 0.0,
    "end_time": 3.5
  }
}

UI Content

{
  "type": "UI",
  "content": {
    "id": 456,
    "text": "Submit Button",
    "timestamp": "2024-03-08T14:30:00Z",
    "app_name": "Safari",
    "window_name": "Login Page",
    "file_path": "/path/to/ui.json",
    "offset_index": 23,
    "browser_url": "https://example.com/login"
  }
}

Response Example

{
  "data": [
    {
      "type": "OCR",
      "content": {
        "frame_id": 12345,
        "text": "authentication code example",
        "timestamp": "2024-03-08T14:30:00Z",
        "app_name": "VS Code",
        "window_name": "auth.ts",
        "focused": true,
        "device_name": "MacBook Pro"
      }
    },
    {
      "type": "Audio",
      "content": {
        "chunk_id": 789,
        "transcription": "We need to implement authentication",
        "timestamp": "2024-03-08T14:25:00Z",
        "device_name": "Microphone",
        "device_type": "Input",
        "speaker": {
          "id": 1,
          "name": "John"
        }
      }
    }
  ],
  "pagination": {
    "limit": 20,
    "offset": 0,
    "total": 2
  }
}

TypeScript SDK Example

const githubActivity = await pipe.queryScreenpipe({
  browserUrl: "github.com",
  contentType: "ocr",
  limit: 20,
  includeFrames: true
});

const searchResults = await pipe.queryScreenpipe({
  q: "authentication",
  browserUrl: "auth0.com",
  appName: "Chrome",
  contentType: "ocr",
  startTime: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString(),
  endTime: new Date().toISOString(),
  limit: 50
});

Error Responses

400 Bad Request

Invalid parameters or malformed query:

{
  "error": "Invalid content_type parameter"
}

500 Internal Server Error

Database or server error:

{
  "error": "Failed to execute search query"
}

Performance Tips

Optimize Your Queries:

• Use specific content_type filters to reduce search scope
• Add time range filters to limit the search window
• Set reasonable limit values (default 20 is optimal)
• Avoid include_frames=true unless necessary
• Use app_name or browser_url filters to narrow results

Next Steps

• Retrieve frame images using frames endpoint
• Monitor API health with health endpoint