x86_64 builds will not trigger

[jeff_glanz]

I have a test project with a reference to a single rhel6 x86_64 repo. I’ve committed source successfully, but the x86_64 will not trigger, nor are there build results. I’ve waited up to 3 days – no build results.

If I look in the UI, I see that the single package in the test project is ‘fine’. But no build result is available.

I’ve tried various approaches:
- Restarting the dispatcher
- Restarting the scheduler and delayed job services
- Reloading updated source
- Rebooted our SCREEN worker hosts
- Rebooted our master api host
- I see references to the test project in /srv/obs/run/bs_sched.x86_64.state
- I do not see references to the test project in /srv/obs/run/dispatch.data
- No jobs are active in /srv/obs/jobs

Here’s the scheduler info for x86_64. This file has not had an update for 20 days:
strings /srv/obs/info/schedulerinfo.x86_64
<schedulerinfo arch="x86_64" started="1482469830" time="1484358355">
<slept>2639</slept>
<notready>8</notready>
<queue high="0" med="0" low="0" next="0" />
<projects>376</projects>
<repositories>760</repositories>
<worst project=".. ... various projects" packages="4" time="4" />
. . .
<buildavg>372.90676985878</buildavg>
<avg>0.235478806907378</avg>
<variance>0.452930764917012</variance>
</schedulerinfo>


I’ve looked in dispatcher and scheduler logs, and can find nothing related to the test project.

Does anyone have info (or a link) on how to diagnose and resolve this issue?? Could the integrity of the underlying mysql database be compromised? Is there a way to force rescheduling?

Thanks for the help!

[jeff_glanz]

We resolved this issue.

We had a project that was linking to a remote obs instance - which another team had transitioned to a different hostname. Unfortunately, this blocked the x86_64 scheduler. After removing that project and its home dependencies, then restarting the schedulers, they unblocked and built all pending packages.