The service.py module#

Summary#

ResourceLimits

Host-level resource constraints enforced by the manager.

JobInfo

Mutable state container for a single simulation.

ResourceMonitor

Async background task that samples host telemetry every N seconds.

JobPoolManager

Priority-aware FIFO queues plus running-set tracker.

JobManager

Async job manager combining resource monitoring and job scheduling.

SchedulerManager

Thin async wrapper around cluster scheduler commands.

JobStatus

Terminal and non-terminal job states used internally and exposed via REST.

submit_job_to_manager

Helper coroutine that submits a job to a remote Job Manager.

main

Example usage of the JobManager class.

logger

Description#

Async job manager with pool scheduling and REST/WebSocket API.

The module implements a fully asynchronous multi-tenant job manager that:

  • enforces host resource limits (CPU, memory, disk, concurrency),

  • maintains priority queues (negative → low, zero → normal, positive → high),

  • exposes REST and Socket.IO endpoints for integration,

  • supports external schedulers (SLURM, LSF, PBS, Windows-HPC) and local subprocess execution,

  • guarantees exactly-once execution and graceful draining on shutdown.

It is designed to be embedded inside a PyEDB process or deployed as a stand-alone micro-service (Docker, systemd, Kubernetes).

Examples#

Stand-alone REST server:

python -m pyedb.workflows.job_manager.service

Embedded inside PyEDB:

from pyedb.workflows.job_manager.service import JobManager, ResourceLimits

manager = JobManager(ResourceLimits(max_concurrent_jobs=4))
await manager.submit_job(config, priority=10)

The REST API is self-documenting at runtime:

curl http://localhost:8080/resources
curl http://localhost:8080/queue
curl -X POST http://localhost:8080/jobs/submit -d @cfg.json

Module detail#

async service.submit_job_to_manager(config: pyedb.workflows.job_manager.backend.job_submission.HFSSSimulationConfig, priority: int = 0, manager_url: str = 'http://localhost:8080') → str#

Helper coroutine that submits a job to a remote Job Manager.

Falls back to local execution if the HTTP call fails (offline mode).

Parameters:
configHFSSSimulationConfig

Validated configuration.

priorityint, optional

Job priority. Default is 0.

manager_urlstr, optional

Base URL of the manager. Default is "http://localhost:8080".

Returns:
str

Job identifier (local or remote).

Raises:
Exception

If both remote and local execution fail.

Notes

This function is useful for clients that want to submit jobs to a remote manager but maintain offline capability.

async service.main()#

Example usage of the JobManager class.

This demonstrates how to create a job manager with custom resource limits and submit jobs with different priorities.

service.logger#