VideoIngestFragmentConsumer

Bases: Thread

Kafka-based threaded consumer for processing VideoIngestFragment messages.

This consumer subscribes to a Kafka topic and dispatches them to per-key input queues managed by the application (e.g. one per camera).

on_new_key is a callback that is invoked when a new key (e.g. camera ID) is encountered. It should return a new input queue for that key, which will be used to enqueue messages for further processing (e.g. decoding and frame extraction).

It periodically pushes per-key ingestion latency metrics to a shared metrics queue, if provided, and supports gracefully stopping the consumer and releasing resources.

Attributes:
  • topic (str) –

    Kafka topic to subscribe to.

  • bootstrap_servers (str) –

    Kafka bootstrap server address.

  • group_id (str) –

    Consumer group ID used by Kafka for partition assignment.

  • on_new_key (Callable[[str], Queue[VideoIngestFragment]]) –

    Function that returns an input queue given a new message key.

  • on_client_stop (Optional[Callable[[int], None]]) –

    Optional callback triggered after consumer shutdown, calls with the consumer index.

  • index (int) –

    Index used for thread naming/logging.

  • metrics_queue (Optional[Queue[VideoIngestFragmentMetric]]) –

    Optional queue where metrics are pushed every 60 seconds.

  • drop_latency_threshold (float) –

    Messages older than this (in seconds) are dropped.