JobPoolManager

class pyedb.workflows.job_manager.backend.service.JobPoolManager(resource_limits: ResourceLimits)

Priority-aware FIFO queues plus running-set tracker.

The implementation is lock-free (uses deque and dict) and async-safe (no awaits, therefore can be invoked from any thread).

Parameters:

resource_limitsResourceLimits

Constraints used by can_start_job().

Attributes:

job_queueDeque[str]

FIFO queue for normal priority jobs

priority_queueDict[int, List[str]]

Priority-based queues (key=priority, value=job_ids)

running_jobsSet[str]

Set of currently running job IDs

job_prioritiesDict[str, int]

Mapping of job_id to priority

Overview

Methods

add_job	Insert job into the appropriate queue (priority or FIFO).
get_next_job	Return the next job to be started (highest priority first).
remove_job	Idempotently remove a job from all queues.
can_start_job	Boolean predicate that decides whether a new job may be started.
get_queue_stats	Real-time snapshot for REST /queue endpoint.

Attributes

resource_limits
job_queue
priority_queue
running_jobs
job_priorities

Import detail

from pyedb.workflows.job_manager.backend.service import JobPoolManager

Attribute detail

JobPoolManager.resource_limits

JobPoolManager.job_queue: Deque[str]

JobPoolManager.priority_queue: Dict[int, List[str]]

JobPoolManager.running_jobs: Set[str]

JobPoolManager.job_priorities: Dict[str, int]

Method detail

JobPoolManager.add_job(job_id: str, priority: int = 0)

Insert job into the appropriate queue (priority or FIFO).

Parameters:

job_idstr

Unique identifier.

priorityint, optional

Negative (low), zero (normal), positive (high). Default is 0.

JobPoolManager.get_next_job() → str | None

Return the next job to be started (highest priority first).

Returns:

str or None

Job identifier or None if all queues are empty.

Notes

Priority queues are checked first (highest to lowest), then the normal FIFO queue.

JobPoolManager.remove_job(job_id: str)

Idempotently remove a job from all queues.

Parameters:

job_idstr

Identifier to purge.

JobPoolManager.can_start_job(resource_monitor: ResourceMonitor) → bool

Boolean predicate that decides whether a new job may be started.

Checks resource limits without violating constraints.

Parameters:

resource_monitorResourceMonitor

Source of current host telemetry.

Returns:

bool

True → job may be started, False → remain queued.

JobPoolManager.get_queue_stats() → Dict[str, Any]

Real-time snapshot for REST /queue endpoint.

Returns:

dict

Queue statistics with keys: - total_queued: Total jobs in all queues - regular_queue_size: Jobs in normal FIFO queue - priority_queues: Dict of priority -> count - running_jobs: Number of currently running jobs - max_concurrent: Maximum concurrent jobs allowed