Skip to content
Reverse Engineering Lab logo Reverse Engineering Lab

RPI Camera Plugin Architecture

An optional plugin that connects camera devices to RELab for remote capture, LL-HLS preview streaming, and YouTube streaming. The backend remains the control plane and default relay path; when the camera and client share a LAN, the RELab app can optionally switch to direct local access for lower-latency preview and capture.

For platform-side setup and day-to-day usage, see the RPI Camera User Guide. For device installation and deployment, see the RPI Camera Plugin repository.

System Diagram

Section titled “System Diagram”
graph TD
    Researcher[Researcher] -->|Capture workflow| FrontendApp[Expo App]
    FrontendApp -->|API requests| MainAPI[FastAPI Backend]


    MainAPI -->|Relay commands + backend APIs| RpiCamAPI[RPI Camera API]
    FrontendApp -->|Optional local direct mode| RpiCamAPI
    RpiCamAPI -->|Controls| Camera[Camera Hardware]


    RpiCamAPI -->|LL-HLS preview| Researcher
    RpiCamAPI -->|RTMP stream| YouTube[YouTube API]
    RpiCamAPI -->|Captured image + metadata| MainAPI


    GoogleOAuth[Google OAuth] -->|User tokens| MainAPI
    MainAPI -->|Livestream management| YouTube


    subgraph DataStores ["Data Stores"]
        Database[(PostgreSQL)]
        FileStorage[(File Storage)]
        Redis[(Redis)]
    end


    MainAPI -->|Store file metadata| Database
    MainAPI -->|Store captured images| FileStorage
    MainAPI -->|Recording session state| Redis

??? info “Styled diagram (ELK layout)“

```mermaid
graph TD
    Researcher["fa:fa-flask Researcher"] -->|Capture workflow| FrontendApp["fa:fa-mobile-screen Expo App"]
    FrontendApp -->|API requests| MainAPI["fa:fa-bolt FastAPI Backend"]


    MainAPI -->|Relay commands + backend APIs| RpiCamAPI["fa:fa-microchip RPI Camera API"]
    FrontendApp -->|Optional local direct mode| RpiCamAPI
    RpiCamAPI -->|Controls| Camera["fa:fa-camera Camera Hardware"]


    RpiCamAPI -->|LL-HLS preview| Researcher
    RpiCamAPI -->|RTMP stream| YouTube["fa:fa-video YouTube API"]
    RpiCamAPI -->|Captured image + metadata| MainAPI


    GoogleOAuth["fa:fa-right-to-bracket Google OAuth"] -->|User tokens| MainAPI
    MainAPI -->|Livestream management| YouTube


    subgraph DataStores ["Data Stores"]
        Database[("fa:fa-database PostgreSQL")]
        FileStorage[("fa:fa-hard-drive File Storage")]
        Redis[("fa:fa-gauge-high Redis")]
    end


    MainAPI -->|Store file metadata| Database
    MainAPI -->|Store captured images| FileStorage
    MainAPI -->|Recording session state| Redis


    classDef actor fill:#e8f4f8,stroke:#2b6cb0,stroke-width:2px,color:#1a365d
    classDef frontend fill:#f0fff4,stroke:#276749,stroke-width:1.5px,color:#1a4731
    classDef backend fill:#fefcbf,stroke:#b7791f,stroke-width:2px,color:#5f4b0a
    classDef datastore fill:#e9d8fd,stroke:#6b46c1,stroke-width:1.5px,color:#44337a
    classDef external fill:#feebc8,stroke:#c05621,stroke-width:1.5px,color:#7b341e
    classDef hardware fill:#e2e8f0,stroke:#4a5568,stroke-width:2px,color:#1a202c


    class Researcher actor
    class FrontendApp frontend
    class MainAPI backend
    class Database,FileStorage,Redis datastore
    classDef auth fill:#fed7e2,stroke:#b83280,stroke-width:1.5px,color:#702459


    class YouTube external
    class GoogleOAuth auth
    class RpiCamAPI,Camera hardware
```

Interaction Flow

Section titled “Interaction Flow”
sequenceDiagram
    participant Researcher
    participant App as Expo App
    participant Backend as Main Backend
    participant YouTubeAPI
    participant RPiCamAPI as Raspberry Pi API
    participant Camera as Camera Hardware


    %% Camera Registration
    Researcher->>App: Enter pairing code
    App->>Backend: Claim pairing code
    Backend->>Backend: Create camera + relay credentials
    Backend-->>RPiCamAPI: Relay credentials via pairing flow
    Backend-->>App: Return camera details
    App-->>Researcher: Show paired camera


    %% Image Capture Flow
    Researcher->>App: Request image capture
    App->>Backend: Request image capture
    Backend->>RPiCamAPI: Forward request via relay
    RPiCamAPI->>Camera: Control camera
    Camera->>RPiCamAPI: Image data
    RPiCamAPI-->>Backend: Image data & metadata
    Backend->>Backend: Store in database & link to product
    Backend-->>App: Return image
    App-->>Researcher: Show image


    %% Optional Local Direct Mode
    Researcher->>App: Open paired camera on same LAN
    App->>Backend: Fetch camera status
    Backend-->>App: Relay state + local bootstrap info
    App->>RPiCamAPI: Optional direct local requests (X-API-Key)
    RPiCamAPI-->>App: Lower-latency preview/capture responses


    %% YouTube Streaming Flow
    Researcher->>App: Start YouTube recording
    App->>Backend: Start YouTube recording
    Backend->>YouTubeAPI: Create live event (OAuth)
    YouTubeAPI-->>Backend: Stream key & broadcast info
    Backend->>RPiCamAPI: Start stream with YouTube config
    RPiCamAPI->>Camera: Start recording
    RPiCamAPI->>YouTubeAPI: Direct stream to YouTube
    Backend->>Backend: Save video record in database
    Backend-->>App: Return video details
    App-->>Researcher: Show video details


    %% Local Preview Stream
    Researcher->>App: Request preview stream
    App->>Backend: Request preview stream
    Backend->>RPiCamAPI: Start preview / mark activity
    RPiCamAPI->>Camera: Start streaming
    RPiCamAPI-->>Backend: Stream info
    Backend-->>App: Stream viewer URL
    App-->>Researcher: Open preview
    Researcher->>RPiCamAPI: Direct HLS requests
    RPiCamAPI-->>Researcher: Stream content


    %% Stop Streaming
    Researcher->>App: Stop recording or preview
    App->>Backend: Stop recording or preview
    Backend->>RPiCamAPI: Stop stream
    RPiCamAPI->>Camera: Stop camera

Key Design Decisions

Section titled “Key Design Decisions”
  • Backend as control plane: Pairing, relay orchestration, capture storage, and YouTube coordination stay in the main backend.
  • Two contract layers: backend OpenAPI remains the public app-facing contract, while a smaller shared private device contract covers pairing payloads, relay envelopes, local-access bootstrap, and Pi-initiated upload acknowledgements.
  • Optional local direct mode: After pairing, the app can switch to LAN-direct access for lower-latency preview and capture when the device is reachable locally.
  • Relay-first registration: Cameras pair through a short-lived code and receive runtime relay credentials from the backend; operators do not manually copy long-lived API keys in the normal path.
  • Media storage: Captured images are stored in RELab’s file storage and linked to the originating product or component record automatically.
  • Optional YouTube integration: Streaming is mediated through the backend’s OAuth connection to YouTube. The device streams directly to YouTube once authorised.

Contract Ownership

Section titled “Contract Ownership”
  • Frontend -> backend uses only backend-owned public routes and schemas.
  • Backend -> plugin uses a smaller private device seam: pairing register/poll payloads, relay command/response envelopes, local-access info, and direct upload/self-unpair acknowledgements.
  • Shared models exist to keep that private seam typed and versioned without leaking plugin implementation details into the frontend contract.