Skip to content

Add /healthcheck endpoint required by cl-mirrors mirrorservice#2

Open
amikhailov7 wants to merge 4 commits into
mainfrom
add-healthcheck-endpoint
Open

Add /healthcheck endpoint required by cl-mirrors mirrorservice#2
amikhailov7 wants to merge 4 commits into
mainfrom
add-healthcheck-endpoint

Conversation

@amikhailov7

Copy link
Copy Markdown
Collaborator

CloudLinux mirrorservice (https://repo.cloudlinux.com/cloudlinux/mirrorlists/cl-mirrors) filters mirrors lacking a fresh /healthcheck endpoint. Mirrors deployed via these playbooks previously registered in the service but never appeared in mirrorlist responses (status 404 on /healthcheck → marked unavailable).

shuiscloud and others added 4 commits June 5, 2026 18:48
CloudLinux mirrorservice (https://repo.cloudlinux.com/cloudlinux/mirrorlists/cl-mirrors)
filters mirrors lacking a fresh /healthcheck endpoint. Mirrors deployed via
these playbooks previously registered in the service but never appeared in
mirrorlist responses (status 404 on /healthcheck → marked unavailable).

This change wires up /healthcheck for all four playbooks (combined-mirror,
complete-swng-rsync, specific-version-rsync, yum-reposync):

1. Installs update-healthcheck.sh which writes /var/www/healthcheck.html in
   the format expected by the mirrorservice
   ("<source> | Status: OK | YYYY/MM/DD HH:MM:SS")
2. Exposes /healthcheck via nginx (HTTP + HTTPS) as a static alias
3. Adds ExecStartPost to the sync service so timestamp refreshes on every
   successful sync run
4. Generates initial /healthcheck during playbook run so the file exists
   before the first sync completes

Timestamp staleness threshold is 12h (mirrorservice config); the sync
schedule of every 4h keeps it well within the window.
Replaces the per-playbook bash script (update-healthcheck.sh) with a
single shared Python tool (ansible/healthcheck/healthcheck_update.py)
re-used from the internal CloudLinux healthcheck repo. License is
Apache 2.0; depends on python3-dotenv (available in EPEL on RHEL-family
and Debian/Ubuntu default repos).

What changes per playbook:
- 2 bash tasks (copy update-healthcheck.sh + ensure dir) become 7 Python
  tasks (epel-release, python3-dotenv, /opt/healthcheck dir, copy py,
  copy .env, /var/www dir, generate initial PENDING HTML).
- service .j2 ExecStartPost now invokes /opt/healthcheck/healthcheck_update.py
  instead of /usr/local/bin/update-healthcheck.sh.
- All 7 healthcheck-related tasks get 'tags: healthcheck' so existing
  deployments can re-run them surgically via:
      ansible-playbook <flavor>/playbook.yml --tags healthcheck
  without touching other infrastructure.

New shared dir ansible/healthcheck/:
- healthcheck_update.py  (the tool, 4217 bytes, mode 0755)
- .env                   (default paths; deploy with force: false)
- LICENSE                (Apache 2.0)

Old per-playbook update-healthcheck.sh files removed.

Validated end-to-end on test-client-repo.atm.cl-mirror.net 2026-06-11:
PF-600 Variant A mirror landed in mirrorlist response position #2
(see Slite contract pp-pscmqW-GDqV, Jira PF-600).
cl-mirrors mirrorservice switched from HTML /healthcheck endpoint to
machine-readable /healthcheck.json endpoint (mirror_processor.py change
deployed on redis.swng.mirrorservice 2026-06-12 by Stepan Oksanichenko).

New expected JSON shape:
  {
    "sync_status": [
      {"repo": "<name>", "status": "OK", "time": "<datetime>"}
    ]
  }

Old HTML endpoint still served alongside for backward compatibility
(future cl-mirrors releases may drop HTML parser entirely).

Changes:
- nginx templates: add 'location = /healthcheck.json' alongside existing
  '/healthcheck' (HTML). +9 location blocks across 4 flavors.
- service .j2 + playbook.yml: ExecStartPost / initial PENDING args
  changed from '--service swng --field source' to '--service sync
  --field repo' so healthcheck_update.py writes block_name='sync_status'
  and field='repo' (matches new contract). +8 lines across 4 flavors.
- defaults/main.yml: add healthcheck_json_file=/var/www/healthcheck.json
  variable (mirrors healthcheck_file/HTML var pattern).

Validated end-to-end on test-client-repo.atm.cl-mirror.net 2026-06-12:
GET /healthcheck.json returns HTTP 200 with correct shape; mirror
landed in checked.swng.mirrors with status='ok' after manual ApplY (PR
contains the full automation).
Migration scenario for existing customers (re-run with --tags healthcheck
on top of an older deployment without /healthcheck endpoint):

1. Existing healthcheck tasks already tagged 'healthcheck' (commit 549aa57)
   deploy /opt/healthcheck/healthcheck_update.py + .env + initial HTML.
2. But nginx /healthcheck.json location + systemd ExecStartPost hook were
   added to TEMPLATE TASKS that were not tagged — so --tags healthcheck
   alone would leave /healthcheck.json 404 (no nginx location) and
   ExecStartPost would never fire (no service unit update).

This commit:
- Tags nginx template tasks (Create Nginx configuration + Create Nginx
  HTTPS configuration + Update HTTP Nginx configuration) with 'healthcheck'
- Tags systemd service template tasks (Create * systemd service *) with
  'healthcheck'
- Adds ExecStartPost healthcheck call to combined-mirror's
  cloudlinux-complete-mirror.service.j2 (was missing — sync_mode=combined
  customers had no healthcheck status updates)

Customer migration command:
  git -C cloudlinux-mirrors pull origin main
  ansible-playbook ansible/<flavor>/playbook.yml --tags healthcheck \
    -e mirror_domain=<their host>

After this, /healthcheck.json works + first sync triggers ExecStartPost
which flips Status to OK.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants