queue

A queue class for the gutscore scheduler.

Classes

Queue

A disk-based SQL queue for the gutscore scheduler.

Module Contents

class Queue(db_name: str = 'guts_queue.db')

A disk-based SQL queue for the gutscore scheduler.

The schduler queue is a lightweight class interfacing with a disk-based SQL queue. It provides the scheduler components (workers, workergroup, resource manager, tasks, …) an atomic process to interact with the flow of GUTS algorithms.

Variables:

_db_name – The queue file name

file_name() → str

Return the queue file name.

Returns:

The queue file name

add_task(task: scheduler.task.Task, deps: uuid.UUID | None = None) → uuid.UUID

Add a new task to the queue.

Parameters:

• task – The task to add to the queue

• deps – The UUID of the task that this task depends on

Returns:

The UUID of the added task

fetch_task() → tuple[int, uuid.UUID, scheduler.task.Task] | None

Fetch the next pending task and mark it as ‘in_progress’.

A BEGIN EXCLUSIVE is nessesary to avoid race conditions from other worker between the select and update.

Returns:

A tuple of (task_id, task_uuid, task) or None

mark_task_done(task_uuid: uuid.UUID) → None

Mark the task as done.

Parameters:

task_uuid – The UUID of the task

increment_completed_tasks() → int

Atomically increment the completed tasks counter and fetch its new value.

Returns:

The new value of the completed tasks counter

get_completed_tasks() → int

Retrieve the current value of the completed tasks counter.

Returns:

The current value of the completed tasks counter

get_running_tasks_count() → int

Return the number of tasks marked in-progress.

Returns:

The number of tasks marked in-progress

get_remaining_tasks_count() → int

Return the number of tasks marked pending/in-progress.

Returns:

The number of tasks marked pending/in-progress

get_tasks_count() → int

Return the total number of tasks in the queue.

Returns:

The total number of tasks

add_event(event: scheduler.event.Event) → None

Add a new event to the queue.

Parameters:

event – The event to add

fetch_event() → tuple[int, int, scheduler.event.Event] | None

Fetch the next pending event.

Returns:

A tuple of (event_id, acc_count, event)

get_events_count() → int

Return the total number of events in the queue.

Returns:

The total number of events

register_worker(wid: tuple[int, int]) → None

Register a worker in the queue.

Parameters:

wid – The worker id, defined by group and worker

unregister_worker(wid: tuple[int, int]) → None

Unregister a worker from the queue.

Parameters:

wid – The worker id, defined by group and worker

update_worker_status(wid: tuple[int, int], status: str) → None

Update the worker status in queue.

Parameters:

• wid – The worker id, defined by group and worker

• status – The worker status

get_workers_count() → int

Return the number of workers.

Returns:

The number of workers

get_active_workers_count() → int

Return the number of active workers.

Returns:

The number of active workers

register_worker_group(gid: int, status: str | None = None) → None

Register a worker group in the queue.

Parameters:

• gid – The worker group id

• status – The worker group status

unregister_worker_group(gid: int) → None

Unregister a worker group from the queue.

Parameters:

gid – The worker group id

check_worker_group(gid: int) → str

Return the status of a given worker group.

Parameters:

gid – The worker group id

Returns:

The worker group status

update_worker_group_status(gid: int, status: str) → None

Update the worker group status in queue.

Parameters:

• gid – The worker group id

• status – The worker group status

update_worker_group_resources(gid: int, resource: str) → None

Update the worker group resources set in queue.

Parameters:

• gid – The worker group id

• resource – The resource set as a json string

get_worker_group_resource(gid: int) → str | None

Query the queue to get a given workergroup resource.

Parameters:

gid – The worker group id

Returns:

The resource set

get_worker_groups_count() → int

Return the number of worker groups.

Returns:

The number of worker groups

delete(timeout: int = 60) → None

Delete the DB when all tasks and workers are done.

Parameters:

• wait_for_done – Wait until all tasks and workers are done

• timeout – Timeout in seconds