Error: Process for Job Will Be Killed

Issue

The following error appears:

Process for job will be killed after its state was changed from RUNNING to RETRY/FAILED.

Environment

• HVR 5
• HVR 6

Resolution

To resolve this issue, determine whether more than one scheduler is running against the same repository. The scheduler starts processes that display in Linux as shown below:

ps -ef | grep hvrsched

oracle    4599  4562  0 19:33 ?  00:00:00 hvrscheduler -h oracle hvrhub/!{Qb/Q.KqR}!

oracle    4600  4599  0 19:33 ?  00:00:00 hvrscheduler -swork -Rhvr_scheduler -horacle hvrhub/!{Qb/Q.KqR}!

oracle    4602  4599  0 19:33 ?  00:00:00 hvrscheduler -slisten -horacle hvrhub/!{Qb/Q.KqR}!

If another scheduler is running on the same machine, you will find two scheduler process trees (6 processes). If the repository is used in another configuration, it could be running on a different machine.

If it is unclear which of the schedulers is the correct scheduler, kill all schedulers found by stopping the scheduler either in the GUI or at the command line.

There are two command line methods for stopping the scheduler:
hvrscheduler -k hvrhub/!{Qb/Q.KqR}! kill -9 4599

Once resolved, restart the applicable scheduler(s).

Example on how to find multiple hub connections in a PostgreSQL hub repository;

On the repository database (PostgreSQL), check for old connections using:

SELECT datname, usename, client_addr, backend_start, state, query FROM pg_stat_activity WHERE client_addr IS NOT NULL ORDER BY backend_start DESC;

If the results show an IP address that doesn't belong to the current hub or the current user running the query, a hub server is still running elsewhere and must be stopped.

Cause

This issue occurs when the scheduler stops the job because the status in the repository has been set to a non-running state, such as FAILED or RETRY.

The scheduler displays this message because the scheduler did not change the status, indicating that more than one scheduler is running against the same repository.