fix few things

This commit is contained in:
victor.kolomin committed 2026-07-06 10:04:21 +02:00
1 parent 8665ea22e1
commit a180583892
13 files changed
+689 -381

No files matched your search

-23
View File
@@ -1,23 +0,0 @@
kind: pipeline
type: docker
name: build-control-container
steps:
- name: build and push
image: plugins/docker
settings:
repo: git.kolspace.cc/victor.kolomin/controlla
registry: git.kolspace.cc
tags:
- ${DRONE_TAG}
dockerfile: Dockerfile
context: .
platforms: linux/amd64
username:
from_secret: registry_user
password:
from_secret: registry_pass
trigger:
event:
- tag
+1
View File
@@ -1,3 +1,4 @@
data/id_rsa
repos/
skipped.yaml
data/known_hosts
+93 -41
View File
@@ -7,13 +7,13 @@ It scans your docker-compose files, checks for new image versions (using SemVer
## Features
- Scans and parses all services and images in docker-compose files (YAML/Jinja2)
- Scans and parses all services and images in docker-compose files (YAML)
- Checks for new image versions (SemVer-aware, with digest fallback)
- Flexible filtering by keywords (e.g., `rc`, `nightly`, `test`, etc.)
- Skip list and global ignore for images (`skipped.yaml`)
- Interactive Telegram notifications (update/ignore buttons)
- Automatic tag update and git push
- Ready-to-use Docker image, CI/CD via DroneCI
- Ready-to-use Docker image, CI/CD via DroneCI / Woodpecker
---
@@ -28,14 +28,21 @@ cd controlla
### 2. Configure
Edit `config.yaml`:
Edit `data/config.yaml`:
```yaml
git:
repo_url: "git@git.kolspace.cc:victor.kolomin/test-homelab.git"
local_repo_path: ./repos/myrepo
repo_url: "git@git.kolspace.cc:youruser/homelab.git"
local_repo_path: ./data/repos/myrepo
branch: "main"
ssh_key_path: "/app/id_rsa"
ssh_key_path: "/app/data/id_rsa"
author_name: "Controlla Bot"
author_email: "bot@controlla.local"
# Optional: GitHub Personal Access Token for authenticated ghcr.io requests.
# Significantly reduces the chance of hitting 429 rate limits.
# Can also be set via the CONTROLLA__GITHUB__TOKEN environment variable.
# github_token: "ghp_yourTokenHere"
checker:
watch_folder: services
@@ -58,7 +65,7 @@ checker:
```yaml
updates:
- image: ghcr.io/lldap/lldap
new_tag: 2024.0.0-01-16
new_tag: v2.0.0
ignored_images:
- ghcr.io/immich-app/postgres
```
@@ -68,11 +75,11 @@ ignored_images:
```sh
docker build -t controlla .
docker run --rm \
-v $(pwd)/config.yaml:/app/data/config.yaml \
-v $(pwd)/skipped.yaml:/app/data/skipped.yaml \
-v $(pwd)/data/config.yaml:/app/data/config.yaml \
-v $(pwd)/data/skipped.yaml:/app/data/skipped.yaml \
-v ~/.ssh/id_rsa:/app/data/id_rsa:ro \
-e TELEGRAM_TOKEN=your_telegram_token \
-e TELEGRAM_CHAT_ID=your_chat_id \
-e CONTROLLA__TELEGRAM__TOKEN=your_telegram_token \
-e CONTROLLA__TELEGRAM__CHAT__ID=your_chat_id \
controlla
```
@@ -82,11 +89,48 @@ docker run --rm \
- Receive notifications about new image versions
- Interactive buttons: "Update", "Ignore"
- `/ignore_image <image>` command — add image to global ignore
- `/ignore_image <image>` command — add image to global ignore list
---
## CI/CD (DroneCI)
## Environment Variables
| Variable | Description |
|---|---|
| `CONTROLLA__TELEGRAM__TOKEN` | Telegram bot token |
| `CONTROLLA__TELEGRAM__CHAT__ID` | Chat ID for notifications |
| `CONTROLLA__DOCKERHUB__USERNAME` | DockerHub username (for private images) |
| `CONTROLLA__DOCKERHUB__PASSWORD` | DockerHub password / token |
| `CONTROLLA__GITHUB__TOKEN` | GitHub PAT for authenticated ghcr.io requests (recommended) |
| `SSH_KEY_PATH` | Path to SSH key inside the container (default: `/app/data/id_rsa`) |
| `SSH_KNOWN_HOSTS` | Path to known_hosts inside the container (default: `/app/data/known_hosts`) |
---
## Fixing 429 Rate Limit Errors on ghcr.io
GHCR enforces strict rate limits for anonymous requests. If you see:
```
Error getting tags for ghcr.io/linuxserver/bookstack: GHCR request failed with status code: 429
```
Do the following:
1. **Create a GitHub Personal Access Token (PAT)** with the `read:packages` scope:
`GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens`
2. **Pass the token** via environment variable:
```sh
-e CONTROLLA__GITHUB__TOKEN=ghp_yourTokenHere
```
or add `github_token: "ghp_yourTokenHere"` to `config.yaml`.
Controlla also implements **automatic retry with backoff**: on a 429 response it reads the `Retry-After` header and waits accordingly (up to 3 retries). If the header is absent, exponential backoff is used: 10 s → 20 s → 40 s.
---
## CI/CD (DroneCI / Woodpecker)
Example `.drone.yml` for automatic Docker image build and push on git tag:
@@ -100,12 +144,11 @@ steps:
image: plugins/docker
settings:
repo: registry.example.com/youruser/controlla
registry: git.kolspace.cc
registry: registry.example.com
tags:
- ${DRONE_TAG}
dockerfile: Dockerfile
context: .
platforms: linux/amd64,linux/arm64
username:
from_secret: registry_user
password:
@@ -116,45 +159,54 @@ trigger:
- tag
```
- The Docker image will be tagged with your git tag (e.g., `v0.1.0`).
---
## Environment Variables
- `CONTROLLA__TELEGRAM__TOKEN` — Telegram bot token
- `CONTROLLA__TELEGRAM__CHAT__ID` — Chat ID for notifications
- `CONTROLLA__DOCKERHUB__USERNAME` — username for private DockerHub access (if needed)
- `CONTROLLA__DOCKERHUB__PASSWORD` — token for private DockerHub access (if needed)
- `SSH_KNOWN_HOSTS` - path in container for known_host file (Default: /app/data/known_hosts)
- `SSH_KEY_PATH` - path in container for ssh key file (Default: /app/data/id_rsa)
---
## Project Structure
```
internal/
checker/ — service/image discovery and parsing
config/ — configuration management
gitrepo/ — git operations
checker/ — docker-compose service/image discovery and parsing
config/ — configuration loading (YAML + env overrides)
gitrepo/ — git operations (clone, pull, push)
notifier/ — Telegram notifications
registry/ — registry clients (dockerhub, ghcr, etc.)
skiplist/ — skiplist and global ignore logic
registry/ — registry clients (dockerhub, ghcr, common)
skiplist/ — skip list and global ignore logic
updater/ — tag update and git push
versionutils/ — version sorting and filtering
main.go — main service and polling logic
versionutils/ — SemVer sorting and filtering
main.go — main service loop and Telegram handler
Dockerfile — container build
.drone.yml — CI/CD pipeline
config.yaml — main config
skipped.yaml — skiplist and ignored images
data/config.yaml — main configuration file
data/skipped.yaml — skip list and ignored images
```
---
## Changelog
### Recent improvements
#### GHCR 429 rate limit fix
- Added **retry with backoff** for all ghcr.io requests: on a 429 response the `Retry-After` header is respected; exponential backoff (10 s → 20 s → 40 s) is used as a fallback. Up to 3 retries.
- Added **GitHub Personal Access Token** support (`github_token` / `CONTROLLA__GITHUB__TOKEN`). Authenticated token requests to ghcr.io have a much higher rate limit than anonymous ones.
- `GetDigest` on the GHCR client now also uses retry logic and proper authentication.
#### HTTP client stability
- Added a **15-second timeout** to `DockerHubClient` and `GhcrClient` — both previously had no timeout (only `CommonRegistryClient` did).
#### Reduced registry pressure
- Added a **2-second delay between image checks** in the main polling loop to avoid sending all requests in a burst.
#### Data race fix
- Added mutex locking to `skiplist.IsImageIgnored` — it was the only method that read shared data without holding the lock.
#### Dependency consistency
- `sort.go` and `main.go` now both use `github.com/Masterminds/semver/v3`. Previously v1 and v3 were mixed in the same binary.
---
## Roadmap
- [ ] Web UI for skiplist management
- [ ] Web UI for skip list management
- [ ] Support for other messengers
- [ ] Extended update analytics
@@ -166,5 +218,5 @@ MIT
---
**Contacts:**
[Git.Kolspace](https://git.kolspace.cc/victor.kolomin/controlla)
**Contacts:**
[Git.Kolspace](https://git.kolspace.cc/victor.kolomin/controlla)
+14 -2
View File
@@ -1,15 +1,27 @@
git:
repo_url: "git@example.git"
repo_url: "ssh://git@git-ssh.kolspace.cc:2222/victor.kolomin/homelab.git"
local_repo_path: ./data/repos/myrepo
branch: "main"
ssh_key_path: "/app/data/id_rsa"
author_name: "Controlla Bot"
author_email: "bot@controlla.local"
# Optional: GitHub Personal Access Token for authenticated ghcr.io requests.
# Significantly reduces the chance of hitting 429 rate limits.
# Can also be set via the CONTROLLA__GITHUB__TOKEN environment variable.
# github_token: "ghp_yourTokenHere"
checker:
watch_folder: services
proposed_tags_count: 3
interval_minutes: 15
skip_list_file: "skipped.yaml"
skip_list_file: "/app/data/skipped.yaml"
# Limit the number of pagination pages fetched from GHCR per image.
# Images like immich-server have thousands of tags across many pages and
# can exhaust the rate-limit quota quickly. 5 pages × 200 tags = 1000 tags,
# which is more than enough to find the latest SemVer version.
# Set to 0 for unlimited (original behaviour).
ghcr_max_pages: 5
ignore_keywords:
- rc
- alpha
+36
View File
@@ -0,0 +1,36 @@
# ssh-keyscan -p 2222 git-ssh.kolspace.cc >> ./data/known_hosts
services:
controlla:
build:
context: .
dockerfile: Dockerfile
container_name: controlla
restart: unless-stopped
environment:
# Required: Telegram bot token (from @BotFather)
CONTROLLA__TELEGRAM__TOKEN: "your_telegram_bot_token"
# Required: Telegram chat ID where notifications will be sent
CONTROLLA__TELEGRAM__CHAT__ID: "your_telegram_chat_id"
# Optional: DockerHub credentials (increases pull rate limits)
# CONTROLLA__DOCKERHUB__USERNAME: "your_dockerhub_username"
# CONTROLLA__DOCKERHUB__PASSWORD: "your_dockerhub_token"
# Optional but recommended: GitHub PAT with read:packages scope
# Prevents 429 rate limit errors from ghcr.io
# CONTROLLA__GITHUB__TOKEN: "ghp_yourGithubToken"
volumes:
# Main config file
- ./data/config.yaml:/app/data/config.yaml:ro
# Skip list (will be created automatically if not present)
- ./data/skipped.yaml:/app/data/skipped.yaml
# SSH private key for git push access (read-only)
- ~/.ssh/controlla:/app/data/id_rsa:ro
# SSH known_hosts to avoid host verification prompts
- ./data/known_hosts:/app/data/known_hosts:ro
+4
View File
@@ -29,6 +29,10 @@ type AppConfig struct {
IntervalMinutes int `mapstructure:"interval_minutes"`
SkipListFile string `mapstructure:"skip_list_file"`
IgnoreKeywords []string `mapstructure:"ignore_keywords"`
// GhcrMaxPages limits how many pagination pages are fetched from GHCR per image.
// Images like immich-server have thousands of tags spread across many pages,
// which exhausts rate-limit quota quickly. Default 0 means unlimited.
GhcrMaxPages int `mapstructure:"ghcr_max_pages"`
} `mapstructure:"checker"`
}
+6 -3
View File
@@ -7,6 +7,7 @@ import (
"io"
"net/http"
"strings"
"time"
)
// DockerTagsResponse represents the JSON response from the Docker Hub tags endpoint.
@@ -32,9 +33,11 @@ type DockerHubClient struct {
func NewDockerHubClient(user, pass string) *DockerHubClient {
return &DockerHubClient{
Client: &http.Client{},
User: user,
Pass: pass,
Client: &http.Client{
Timeout: 15 * time.Second,
},
User: user,
Pass: pass,
}
}
+164 -59
View File
@@ -5,64 +5,124 @@ import (
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
"github.com/sirupsen/logrus"
)
const (
ghcrMaxRetries = 3
ghcrRetryBaseWait = 10 * time.Second
)
type GhcrClient struct {
Client *http.Client
Client *http.Client
GithubToken string // optional; if set, used for authenticated token requests
}
func NewGhcrClient() *GhcrClient {
func NewGhcrClient(githubToken string) *GhcrClient {
return &GhcrClient{
Client: &http.Client{},
Client: &http.Client{
Timeout: 15 * time.Second,
},
GithubToken: githubToken,
}
}
// GetTags retrieves all image tags for a given image, handling pagination.
func (c *GhcrClient) GetTags(image string) ([]string, error) {
// Request an anonymous token
authURL := fmt.Sprintf("https://ghcr.io/token?service=ghcr.io&scope=repository:%s:pull", image)
req, err := http.NewRequest(http.MethodGet, authURL, nil)
// doWithRetry executes an HTTP request and retries up to ghcrMaxRetries times
// on 429 Too Many Requests, honouring the Retry-After header when present.
func (c *GhcrClient) doWithRetry(req *http.Request) (*http.Response, error) {
// We need to be able to replay the request on retry. Since these are all
// GET requests with no body, we only need to clone the headers/URL.
url := req.URL.String()
method := req.Method
headerCopy := req.Header.Clone()
var resp *http.Response
var err error
for attempt := 0; attempt <= ghcrMaxRetries; attempt++ {
if attempt > 0 {
// Rebuild the request for subsequent attempts so it is fresh.
req, err = http.NewRequest(method, url, nil)
if err != nil {
return nil, fmt.Errorf("failed to rebuild request: %w", err)
}
req.Header = headerCopy.Clone()
}
resp, err = c.Client.Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusTooManyRequests {
return resp, nil
}
// 429 — parse Retry-After and wait before the next attempt.
resp.Body.Close()
if attempt == ghcrMaxRetries {
break
}
wait := retryAfterDuration(resp.Header.Get("Retry-After"), ghcrRetryBaseWait, attempt)
time.Sleep(wait)
}
return nil, fmt.Errorf("GHCR request failed with status code: %d (rate limited after %d retries)", http.StatusTooManyRequests, ghcrMaxRetries)
}
// retryAfterDuration parses a Retry-After header value (seconds or HTTP-date)
// and returns the sleep duration. Falls back to exponential backoff when the
// header is absent or unparseable.
func retryAfterDuration(header string, base time.Duration, attempt int) time.Duration {
if header != "" {
if secs, err := strconv.Atoi(header); err == nil {
return time.Duration(secs) * time.Second
}
if t, err := http.ParseTime(header); err == nil {
d := time.Until(t)
if d > 0 {
return d
}
}
}
// Exponential backoff: base * 2^attempt
return base * (1 << uint(attempt))
}
// GetTags retrieves image tags for a given image, handling pagination.
// maxPages limits the number of pages fetched; 0 means unlimited.
// For images with thousands of tags (e.g. immich-server) set this to a small
// value (e.g. 5) to avoid burning through registry rate-limit quota.
func (c *GhcrClient) GetTags(image string, maxPages int) ([]string, error) {
token, err := c.getToken(image)
if err != nil {
return nil, fmt.Errorf("failed to create auth request: %w", err)
return nil, err
}
resp, err := c.Client.Do(req)
if err != nil {
return nil, fmt.Errorf("failed to get GHCR token: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("GHCR token request failed with status %d: %s", resp.StatusCode, string(body))
}
var authData struct {
Token string `json:"token"`
}
if err := json.NewDecoder(resp.Body).Decode(&authData); err != nil {
return nil, fmt.Errorf("failed to decode auth response: %w", err)
}
token := authData.Token
// Step 2: Use the token to get the tags list with pagination
var allTags []string
url := fmt.Sprintf("https://ghcr.io/v2/%s/tags/list?n=200", image)
page := 0
for {
req, err = http.NewRequest(http.MethodGet, url, nil)
page++
if maxPages > 0 && page > maxPages {
logrus.Debugf("GHCR: reached max_pages limit (%d) for %s, stopping pagination.", maxPages, image)
break
}
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
return nil, fmt.Errorf("failed to create tags request: %w", err)
}
req.Header.Set("Authorization", "Bearer "+token)
resp, err = c.Client.Do(req)
resp, err := c.doWithRetry(req)
if err != nil {
return nil, fmt.Errorf("failed to get GHCR tags: %w", err)
}
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, fmt.Errorf("GHCR request failed with status code: %d", resp.StatusCode)
return nil, err
}
var result map[string]json.RawMessage
@@ -78,7 +138,6 @@ func (c *GhcrClient) GetTags(image string) ([]string, error) {
}
allTags = append(allTags, tags...)
// Check for pagination
link := resp.Header.Get("Link")
if link == "" || !containsRelNext(link) {
break
@@ -87,7 +146,6 @@ func (c *GhcrClient) GetTags(image string) ([]string, error) {
if nextURL == "" {
break
}
// Ensure the nextURL is absolute
if strings.HasPrefix(nextURL, "/") {
nextURL = "https://ghcr.io" + nextURL
}
@@ -96,14 +154,80 @@ func (c *GhcrClient) GetTags(image string) ([]string, error) {
return allTags, nil
}
// getToken requests an anonymous (or authenticated) pull token for image.
func (c *GhcrClient) getToken(image string) (string, error) {
authURL := fmt.Sprintf("https://ghcr.io/token?service=ghcr.io&scope=repository:%s:pull", image)
req, err := http.NewRequest(http.MethodGet, authURL, nil)
if err != nil {
return "", fmt.Errorf("failed to create auth request: %w", err)
}
// Using a GitHub personal access token raises the rate limit and allows
// access to packages visible to that account.
// GHCR expects Basic auth with the literal username "token" and the PAT as password.
if c.GithubToken != "" {
req.SetBasicAuth("token", c.GithubToken)
}
resp, err := c.doWithRetry(req)
if err != nil {
return "", fmt.Errorf("failed to get GHCR token: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return "", fmt.Errorf("GHCR token request failed with status %d: %s", resp.StatusCode, string(body))
}
var authData struct {
Token string `json:"token"`
}
if err := json.NewDecoder(resp.Body).Decode(&authData); err != nil {
return "", fmt.Errorf("failed to decode auth response: %w", err)
}
return authData.Token, nil
}
func (c *GhcrClient) GetDigest(image, tag string) (string, error) {
// Need a fresh token scoped to this image for the manifest request.
token, err := c.getToken(image)
if err != nil {
// Fall back to unauthenticated if token fetch fails.
token = ""
}
url := fmt.Sprintf("https://ghcr.io/v2/%s/manifests/%s", image, tag)
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
return "", err
}
req.Header.Set("Accept", "application/vnd.docker.distribution.manifest.v2+json")
if token != "" {
req.Header.Set("Authorization", "Bearer "+token)
}
resp, err := c.doWithRetry(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return "", fmt.Errorf("digest request failed: %s", string(body))
}
return resp.Header.Get("Docker-Content-Digest"), nil
}
// containsRelNext checks if the Link header contains rel="next"
func containsRelNext(link string) bool {
return strings.Contains(link, `rel="next"`)
}
// parseNextURL extracts the next page URL from the Link header
// parseNextURL extracts the next page URL from the Link header.
// Example: <https://ghcr.io/v2/immich-app/immich-server/tags/list?next_page_token=...&n=200>; rel="next"
func parseNextURL(link string) string {
// Example: <https://ghcr.io/v2/immich-app/immich-server/tags/list?next_page_token=...&n=200>; rel="next"
parts := strings.Split(link, ";")
if len(parts) < 2 {
return ""
@@ -112,22 +236,3 @@ func parseNextURL(link string) string {
urlPart = strings.Trim(urlPart, "<>")
return urlPart
}
func (c *GhcrClient) GetDigest(image, tag string) (string, error) {
url := fmt.Sprintf("https://ghcr.io/v2/%s/manifests/%s", image, tag)
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
return "", err
}
req.Header.Set("Accept", "application/vnd.docker.distribution.manifest.v2+json")
resp, err := c.Client.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return "", fmt.Errorf("digest request failed: %s", string(body))
}
return resp.Header.Get("Docker-Content-Digest"), nil
}
+3
View File
@@ -94,6 +94,9 @@ func (sl *SkipList) IsSkipped(image, newTag string) bool {
// Check is if an image is in the ignored images list.
func (sl *SkipList) IsImageIgnored(image string) bool {
sl.Lock()
defer sl.Unlock()
for _, img := range sl.data.IgnoredImages {
if img == image {
return true
+81
View File
@@ -0,0 +1,81 @@
package versionutils
import (
"strings"
"testing"
"github.com/Masterminds/semver/v3"
)
func TestTryNormalizeTagForSemVer(t *testing.T) {
cases := []struct {
input string
wantParseable bool
}{
{"1.19.2", true},
{"v1.19.2", true},
{"ee-1.19.2", true},
{"lts-3.0.1", true},
{"v-2.0.0", true},
{"1.2.3.4", true},
{"latest", false},
// "26.03.20250318" is technically valid SemVer but should be filtered
// out by isWeirdTag (bigNumRegexp + year check), so we don't test it here.
}
for _, tc := range cases {
norm := TryNormalizeTagForSemVer(tc.input)
_, err := semver.NewVersion(strings.TrimPrefix(norm, "v"))
got := err == nil
if got != tc.wantParseable {
t.Errorf("TryNormalizeTagForSemVer(%q) -> %q, parseable=%v, want %v",
tc.input, norm, got, tc.wantParseable)
}
}
}
func TestSortAndFilterTagsWithPrefix(t *testing.T) {
tags := []string{"ee-1.19.1", "ee-1.19.2", "ee-1.19.3"}
result := SortAndFilterTags(tags, 3, []string{})
if len(result) == 0 {
t.Fatal("expected at least one result, got none")
}
if result[0] != "ee-1.19.3" {
t.Errorf("expected first result to be ee-1.19.3, got %s", result[0])
}
}
func TestSortAndFilterTagsCollision(t *testing.T) {
// When both "ee-1.19.2" and "1.19.2" are present, "1.19.2" (exact SemVer)
// should win over the normalised "ee-1.19.2".
tags := []string{"ee-1.19.2", "1.19.2", "ee-1.19.3"}
result := SortAndFilterTags(tags, 3, []string{})
for _, tag := range result {
if tag == "ee-1.19.2" {
t.Errorf("expected ee-1.19.2 to be replaced by 1.19.2, got results: %v", result)
}
}
}
func TestSortAndFilterTagsDateTagsFiltered(t *testing.T) {
// Date-like tags (2023.10.05), build-timestamp tags (26.03.20250318),
// and date-dash tags (2026-05-26-debian-rootless) must never appear in output.
tags := []string{
"2023.10.05",
"26.03.20250318",
"2026-05-26-debian-rootless",
"2026-05-26-debian",
"2026-05-26",
"1.19.2",
"1.19.1",
}
result := SortAndFilterTags(tags, 5, []string{})
badTags := []string{"2023.10.05", "26.03.20250318", "2026-05-26-debian-rootless", "2026-05-26-debian", "2026-05-26"}
for _, bad := range badTags {
for _, got := range result {
if got == bad {
t.Errorf("tag %q should have been filtered out, got: %v", bad, result)
}
}
}
}
+61 -14
View File
@@ -6,21 +6,35 @@ import (
"strconv"
"strings"
"github.com/Masterminds/semver"
"github.com/Masterminds/semver/v3"
)
var (
// Exact date tag with dots: 2023.10.05
dateTagRegexp = regexp.MustCompile(`^\d{4}\.\d{2}\.\d{2}$`)
bigNumRegexp = regexp.MustCompile(`^\d{7,}`)
// Date tag with dashes (with optional suffix): 2026-05-26, 2026-05-26-debian-rootless
dateDashTagRegexp = regexp.MustCompile(`^\d{4}-\d{2}-\d{2}`)
// Tag that starts with a very large number (7+ digits), e.g. build timestamps.
bigNumRegexp = regexp.MustCompile(`^\d{7,}`)
// Four-part numeric tag: X.Y.Z.A — explicitly allowed (normalised to X.Y.Z-rev.A).
fourPartTagRegex = regexp.MustCompile(`^\d+\.\d+\.\d+\.\d+$`)
// A segment containing 7+ consecutive digits — catches build timestamps
// embedded in any part of the tag, e.g. "26.03.20250318" or "1.0.20231005".
embeddedTimestampRegexp = regexp.MustCompile(`\b\d{7,}\b`)
)
// Deduplicate, sort and filter tags.
// It removes duplicates, ignores tags with specified keywords, filters out weird tags (dates, big numbers),
// sorts them in descending order and returns the top N tags.
// It preserves the original tag format (with 'v' prefix if present).
// SortAndFilterTags deduplicates, sorts and filters tags.
// It removes duplicates, ignores tags with specified keywords, filters out
// weird tags (dates, big numbers), sorts them in descending SemVer order and
// returns the top N tags.
//
// Collision handling: when two tags normalise to the same SemVer string
// (e.g. "ee-1.19.2" and "1.19.2" both → "1.19.2"), the exact-match tag is
// preferred over the normalised one so that the version being tracked stays
// consistent.
func SortAndFilterTags(tags []string, n int, ignoreKeywords []string) []string {
uniqueVersions := make(map[string]*semver.Version)
tagMap := make(map[string]string) // semver.String() -> original tag
tagMap := make(map[string]string) // semver key → original tag
for _, tag := range tags {
if IsIgnoredTag(tag, ignoreKeywords) {
@@ -30,15 +44,27 @@ func SortAndFilterTags(tags []string, n int, ignoreKeywords []string) []string {
continue
}
// --- MODIFICATION: Try to normalize tag before parsing as SemVer ---
normalizedTag := TryNormalizeTagForSemVer(tag)
cleanTag := strings.TrimPrefix(normalizedTag, "v")
v, err := semver.NewVersion(cleanTag)
// --- END MODIFICATION ---
if err != nil {
continue
}
if err == nil {
uniqueVersions[v.String()] = v
tagMap[v.String()] = tag // сохраняем оригинальный тег!
key := v.String()
if existing, seen := tagMap[key]; seen {
// Prefer the tag that doesn't need normalisation (i.e. is already
// a valid SemVer or just has a "v" prefix).
existingNorm := TryNormalizeTagForSemVer(existing)
tagNorm := TryNormalizeTagForSemVer(tag)
if existingNorm != existing && tagNorm == tag {
// Current tag is "cleaner" — replace.
tagMap[key] = tag
}
// Otherwise keep what we already have.
} else {
uniqueVersions[key] = v
tagMap[key] = tag
}
}
@@ -107,11 +133,31 @@ func IsIgnoredTag(tag string, ignoreKeywords []string) bool {
}
func isWeirdTag(tag string) bool {
// Filter out date-like tags (e.g., 2023.10.05) and big number tags (e.g., 9799770991.0.0)
// as they are unlikely to be valid SemVer tags.
// Explicitly allow X.Y.Z.A tags — they are normalised to X.Y.Z-rev.A by
// TryNormalizeTagForSemVer and must not be discarded here.
if fourPartTagRegex.MatchString(tag) {
return false
}
// Filter out pure date tags like 2023.10.05.
if dateTagRegexp.MatchString(tag) {
return true
}
// Filter out date-prefixed tags like 2026-05-26, 2026-05-26-debian-rootless.
if dateDashTagRegexp.MatchString(tag) {
return true
}
// Filter out tags containing a build timestamp anywhere in them,
// e.g. "26.03.20250318", "1.0.20231005000000". A 7+ digit numeric segment
// is a strong signal for a CI timestamp rather than a real version number.
if embeddedTimestampRegexp.MatchString(tag) {
return true
}
// Filter out tags that start with a very large number (7+ digits leading)
// and whose first segment exceeds the plausible year range.
if bigNumRegexp.MatchString(tag) {
parts := strings.Split(tag, ".")
if len(parts) > 0 {
@@ -120,5 +166,6 @@ func isWeirdTag(tag string) bool {
}
}
}
return false
}
+23 -9
View File
@@ -80,20 +80,25 @@ func NormalizeTag(tag string) string {
return strings.TrimPrefix(tag, "v")
}
// TryNormalizeTagForSemVer attempts to normalize a non-standard tag into a SemVer-compatible format.
// For example, "1.2.3.4" might become "1.2.3-rev.4".
// This is a heuristic approach for specific non-SemVer patterns.
// TryNormalizeTagForSemVer attempts to normalize a non-standard tag into a
// SemVer-compatible format so it can be compared and sorted.
// The original tag string is always preserved separately — normalization is
// only used for version arithmetic, never for display or git operations.
//
// Supported patterns:
// - "1.2.3.4" → "1.2.3-rev.4" (four-part numeric)
// - "ee-1.2.3" → "1.2.3" (arbitrary letter prefix separated by "-")
// - "v-1.2.3" → "1.2.3" (same, with "v" prefix variant)
func TryNormalizeTagForSemVer(tag string) string {
// If it already parses as SemVer, no need to normalize
// Already valid SemVer — nothing to do.
if _, err := semver.NewVersion(strings.TrimPrefix(tag, "v")); err == nil {
return tag
}
// Pattern: X.Y.Z.A (e.g., 1.2.3.4)
// Pattern: X.Y.Z.A (e.g. "1.2.3.4", "26.03.20250318")
parts := strings.Split(tag, ".")
if len(parts) == 4 {
// Try to form X.Y.Z-rev.A
if _, err := strconv.Atoi(parts[3]); err == nil { // Check if the last part is a number
if _, err := strconv.Atoi(parts[3]); err == nil {
normalized := fmt.Sprintf("%s.%s.%s-rev.%s", parts[0], parts[1], parts[2], parts[3])
if _, err := semver.NewVersion(normalized); err == nil {
logrus.Debugf("Normalized tag '%s' to '%s' for SemVer parsing.", tag, normalized)
@@ -102,7 +107,16 @@ func TryNormalizeTagForSemVer(tag string) string {
}
}
// Add other normalization patterns here if needed
// Pattern: <prefix>-X.Y.Z (e.g. "ee-1.19.2", "lts-3.0.1", "v-2.0.0")
// Strip everything up to and including the first "-" and check if the
// remainder is valid SemVer.
if idx := strings.Index(tag, "-"); idx > 0 {
candidate := tag[idx+1:]
if _, err := semver.NewVersion(strings.TrimPrefix(candidate, "v")); err == nil {
logrus.Debugf("Normalized tag '%s' to '%s' for SemVer parsing (prefix stripped).", tag, candidate)
return candidate
}
}
return tag // Return original tag if no normalization helps
return tag
}
+203 -230
View File
@@ -24,7 +24,7 @@ import (
"syscall"
"time"
"github.com/Masterminds/semver"
"github.com/Masterminds/semver/v3"
tgbotapi "github.com/go-telegram-bot-api/telegram-bot-api/v5"
"github.com/sirupsen/logrus"
)
@@ -43,18 +43,17 @@ var cacheMutex sync.Mutex
// sentNotifications stores a record of notifications already sent to Telegram
// to prevent sending duplicates for the same image/tag combination.
var sentNotifications = make(map[string]bool)
var sentNotificationsMutex sync.Mutex // Add mutex for thread-safe access
var sentNotificationsMutex sync.Mutex
// For hash-mode needed map hash → {fullImageName, tag}
var callbackHashMap = make(map[string][2]string) // [fullImageName, tag]
// For hash-mode: maps sha1 hash → {fullImageName, tag}
var callbackHashMap = make(map[string][2]string)
// Decode callback data
func decodeCallbackData(data, prefix string) (fullImageName, tag string, ok bool) {
if !strings.HasPrefix(data, prefix+":") {
return "", "", false
}
enc := strings.TrimPrefix(data, prefix+":")
if strings.HasPrefix(enc, "h:") && len(enc) == 42 { // "h:" + 40 hex
if strings.HasPrefix(enc, "h:") && len(enc) == 42 { // "h:" + 40 hex chars
hash := strings.TrimPrefix(enc, "h:")
val, ok := callbackHashMap[hash]
if !ok {
@@ -85,12 +84,10 @@ func handleCallbackQuery(bot *tgbotapi.BotAPI, callback *tgbotapi.CallbackQuery,
logrus.Infof("Received update request for image: %s, tag: %s", fullImageName, newTag)
// Remove from sentNotifications when acted upon ---
sentNotificationsMutex.Lock()
delete(sentNotifications, fullImageName+":"+newTag)
sentNotificationsMutex.Unlock()
// Looking for servies to update
repoPath := appConfig.Git.LocalRepoPath
relevantFiles, err := checker.FindRelevantFiles(repoPath, appConfig.Checker.WatchFolder)
if err != nil {
@@ -113,10 +110,8 @@ func handleCallbackQuery(bot *tgbotapi.BotAPI, callback *tgbotapi.CallbackQuery,
}
if len(servicesToUpdate) > 0 {
// Update all found services
logrus.Infof("Found %d services to update for image %s", len(servicesToUpdate), fullImageName)
for _, service := range servicesToUpdate {
// Проверяем, совпадает ли уже тег
if service.ImageInfo.Tag == newTag {
logrus.Infof("Service %s in file %s already has tag %s, skipping update.", service.ServiceName, service.FilePath, newTag)
notifier.SendTelegramMessage(bot, callback.Message.Chat.ID, fmt.Sprintf("ℹ️ Service %s already has version %s, no update needed.", service.ServiceName, newTag))
@@ -149,7 +144,6 @@ func handleCallbackQuery(bot *tgbotapi.BotAPI, callback *tgbotapi.CallbackQuery,
logrus.Infof("Received ignore request for image: %s, version: %s", fullImageName, tagToIgnore)
// Remove from sentNotifications when acted upon ---
sentNotificationsMutex.Lock()
delete(sentNotifications, fullImageName+":"+tagToIgnore)
sentNotificationsMutex.Unlock()
@@ -161,14 +155,14 @@ func handleCallbackQuery(bot *tgbotapi.BotAPI, callback *tgbotapi.CallbackQuery,
logrus.Infof("Update for %s with tag %s has been added to the skip list.", fullImageName, tagToIgnore)
}
}
// Clear ALL related notifications/warnings for this image ---
// Clear all cached notifications for this image after any user action.
var fullImageName string
if strings.HasPrefix(callbackData, "update_to:") {
fullImageName, _, _ = decodeCallbackData(callbackData, "update_to")
} else if strings.HasPrefix(callbackData, "ignore:") {
fullImageName, _, _ = decodeCallbackData(callbackData, "ignore")
}
if fullImageName != "" {
sentNotificationsMutex.Lock()
for key := range sentNotifications {
@@ -177,10 +171,10 @@ func handleCallbackQuery(bot *tgbotapi.BotAPI, callback *tgbotapi.CallbackQuery,
}
}
sentNotificationsMutex.Unlock()
logrus.Debugf("Cleared all pending notifications/warnings for %s after user action.", fullImageName)
logrus.Debugf("Cleared all pending notifications for %s after user action.", fullImageName)
}
// Send confirmation to user and delete original message
// Delete the original Telegram message.
deleteConfig := tgbotapi.NewDeleteMessage(callback.Message.Chat.ID, callback.Message.MessageID)
_, err := bot.Request(deleteConfig)
if err != nil {
@@ -188,24 +182,19 @@ func handleCallbackQuery(bot *tgbotapi.BotAPI, callback *tgbotapi.CallbackQuery,
}
}
// Main entry point for the controlla application.
// This service checks container images for updates and notifies via Telegram.
func main() {
// Parse command-line flags
configPath := flag.String("config-file", "./data/config.yaml", "Path to config file")
flag.Parse()
// Set up logging format and level
logrus.SetFormatter(&logrus.TextFormatter{FullTimestamp: true})
logrus.SetLevel(logrus.InfoLevel)
// Load application configuration from YAML file
appConfig, err := config.LoadConfig(*configPath)
if err != nil {
log.Fatalf("Error loading application configuration: %v", err)
}
logrus.Infof("Configuration loaded successfully.")
// Initialize and load skip list for ignored updates
sl := skiplist.NewSkipList(appConfig.Checker.SkipListFile)
if err := sl.Load(); err != nil {
logrus.Errorf("Error loading skip list: %v", err)
@@ -213,7 +202,7 @@ func main() {
logrus.Infof("Skip list loaded successfully.")
}
// Remove old repository clone if exists
// Remove old repository clone if exists.
if _, err := os.Stat(appConfig.Git.LocalRepoPath); err == nil {
logrus.Infof("Removing old repository clone...")
if err := os.RemoveAll(appConfig.Git.LocalRepoPath); err != nil {
@@ -221,27 +210,22 @@ func main() {
}
}
// Set up context for graceful shutdown
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Handle OS signals for shutdown
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigChan
logrus.Infof("Shutting down...")
cancel()
}()
// Initialize Telegram bot
bot, err := tgbotapi.NewBotAPI(appConfig.TelegramToken)
if err != nil {
log.Fatalf("Failed to create Telegram bot: %v", err)
}
// Start Telegram listener in a separate goroutine
go func() {
logrus.Infof("Telegram listener started...")
u := tgbotapi.NewUpdate(0)
@@ -270,18 +254,16 @@ func main() {
logrus.Infof("Telegram listener is shutting down.")
}()
// Set up periodic check interval
checkInterval := time.Duration(appConfig.Checker.IntervalMinutes) * time.Minute
ticker := time.NewTicker(checkInterval)
defer ticker.Stop()
// Initizle registry clients
dockerHubClient := registry.NewDockerHubClient(appConfig.DockerUser, appConfig.DockerPass)
ghcrClient := registry.NewGhcrClient()
commonClients := make(map[string]*registry.CommonRegistryClient) // Created on-the-fly
ghcrClient := registry.NewGhcrClient(appConfig.GithubToken)
commonClients := make(map[string]*registry.CommonRegistryClient)
runCheck(ctx, appConfig, bot, sl, dockerHubClient, ghcrClient, commonClients)
// Main loop: periodic checks
for {
select {
case <-ctx.Done():
@@ -292,7 +274,7 @@ func main() {
}
}
// runCheck checks all container images for updates and sends notifications if updates are available.
// runCheck checks all container images for updates and sends Telegram notifications.
func runCheck(ctx context.Context, appConfig *config.AppConfig, bot *tgbotapi.BotAPI, sl *skiplist.SkipList, dockerHubClient *registry.DockerHubClient, ghcrClient *registry.GhcrClient, commonClients map[string]*registry.CommonRegistryClient) {
logrus.Infof("Starting image update check...")
@@ -314,15 +296,13 @@ func runCheck(ctx context.Context, appConfig *config.AppConfig, bot *tgbotapi.Bo
}
logrus.Infof("Repository cloned or pulled successfully.")
// Get list of relevant files in the repository
relevantFiles, err := checker.FindRelevantFiles(repoPath, appConfig.Checker.WatchFolder)
if err != nil {
logrus.Errorf("Error finding relevant files: %v", err)
return
}
logrus.Infof("Found %d relevant files in watch folder '%s'.", len(relevantFiles), appConfig.Checker.WatchFolder) // Debug log
logrus.Infof("Found %d relevant files in watch folder '%s'.", len(relevantFiles), appConfig.Checker.WatchFolder)
// Parse relevant files to extract services and their images
services, err := checker.ParseRelevantFiles(relevantFiles)
if err != nil {
logrus.Errorf("Error parsing images: %v", err)
@@ -330,7 +310,7 @@ func runCheck(ctx context.Context, appConfig *config.AppConfig, bot *tgbotapi.Bo
}
logrus.Infof("Found a total of %d services.", len(services))
// Group services by full image name to avoid duplicate checks
// Group services by full image name to avoid duplicate registry calls.
groupedServices := make(map[string][]checker.ServiceInfo)
for _, service := range services {
fullImageName := registry.GetFullImageName(&service.ImageInfo)
@@ -338,226 +318,220 @@ func runCheck(ctx context.Context, appConfig *config.AppConfig, bot *tgbotapi.Bo
}
logrus.Infof("Found a total of %d unique images", len(groupedServices))
// Check each unique image for updates
// A small sleep between requests reduces burst pressure on registries.
const interRequestDelay = 2 * time.Second
for fullImageName, services := range groupedServices {
select {
case <-ctx.Done():
return
default:
currentImageTag := services[0].ImageInfo.Tag
logrus.Infof("Parsing image: Registry=%s, Image=%s, Tag=%s", services[0].ImageInfo.Registry, services[0].ImageInfo.Image, currentImageTag)
}
// Clear notifications if image is skipped/ignored ---
if sl.IsSkipped(fullImageName, currentImageTag) || sl.IsImageIgnored(fullImageName) {
if sl.IsSkipped(fullImageName, currentImageTag) {
logrus.Infof("Skipping update for %s. It's in the skip list.", fullImageName)
} else {
logrus.Infof("Image %s is globally ignored (skipped.yaml), skipping.", fullImageName)
}
sentNotificationsMutex.Lock()
for key := range sentNotifications {
if strings.HasPrefix(key, fullImageName+":") {
delete(sentNotifications, key)
}
time.Sleep(interRequestDelay)
currentImageTag := services[0].ImageInfo.Tag
logrus.Infof("Parsing image: Registry=%s, Image=%s, Tag=%s", services[0].ImageInfo.Registry, services[0].ImageInfo.Image, currentImageTag)
// Skip images that use the "latest" tag — there is no meaningful version
// to compare against, so checking them just wastes registry quota.
if currentImageTag == "latest" {
logrus.Infof("Skipping %s: tag is 'latest', version tracking is not possible.", fullImageName)
continue
}
// Skip images on the skip/ignore list.
if sl.IsSkipped(fullImageName, currentImageTag) || sl.IsImageIgnored(fullImageName) {
if sl.IsSkipped(fullImageName, currentImageTag) {
logrus.Infof("Skipping update for %s. It's in the skip list.", fullImageName)
} else {
logrus.Infof("Image %s is globally ignored (skipped.yaml), skipping.", fullImageName)
}
sentNotificationsMutex.Lock()
for key := range sentNotifications {
if strings.HasPrefix(key, fullImageName+":") {
delete(sentNotifications, key)
}
}
sentNotificationsMutex.Unlock()
continue
}
var tags []string
registryName := services[0].ImageInfo.Registry
switch registryName {
case "docker.io":
tags, err = dockerHubClient.GetTags(services[0].ImageInfo.Image)
case "ghcr.io":
tags, err = ghcrClient.GetTags(services[0].ImageInfo.Image, appConfig.Checker.GhcrMaxPages)
default:
client, ok := commonClients[registryName]
if !ok {
logrus.Debugf("Creating new common client for registry: %s", registryName)
client = registry.NewCommonRegistryClient(registryName)
commonClients[registryName] = client
}
tags, err = client.GetTags(services[0].ImageInfo.Image)
}
logrus.Debugf("Available tags for %s: %v", fullImageName, tags)
if err != nil {
logrus.Errorf("Error getting tags for %s: %v", fullImageName, err)
notificationKey := fullImageName + ":error_getting_tags"
sentNotificationsMutex.Lock()
if sentNotifications[notificationKey] {
sentNotificationsMutex.Unlock()
continue
}
sentNotifications[notificationKey] = true
sentNotificationsMutex.Unlock()
chatID, _ := strconv.ParseInt(appConfig.TelegramChatID, 10, 64)
notifier.SendTelegramMessage(bot, chatID, fmt.Sprintf("❌ Error getting tags for %s: %v", fullImageName, err))
continue
}
var tags []string
var err error
registryName := services[0].ImageInfo.Registry
// Successful fetch — clear any previously cached error notification so it
// can fire again if the image breaks in a future cycle.
sentNotificationsMutex.Lock()
delete(sentNotifications, fullImageName+":error_getting_tags")
sentNotificationsMutex.Unlock()
switch registryName {
case "docker.io":
tags, err = dockerHubClient.GetTags(services[0].ImageInfo.Image)
case "ghcr.io":
tags, err = ghcrClient.GetTags(services[0].ImageInfo.Image)
default:
// Create client on-the-fly ---
client, ok := commonClients[registryName]
if !ok {
logrus.Debugf("Creating new common client for registry: %s", registryName)
client = registry.NewCommonRegistryClient(registryName)
commonClients[registryName] = client
}
tags, err = client.GetTags(services[0].ImageInfo.Image)
}
// Filter and sort tags by SemVer, return the top N candidates.
latestTags := versionutils.SortAndFilterTags(tags, appConfig.Checker.ProposedTagsCount, appConfig.Checker.IgnoreKeywords)
if len(latestTags) == 0 {
logrus.Warnf("⚠️ Can't check updates for %s. Current tag: %s is not a SemVer tag. Available tags: %v",
fullImageName, currentImageTag, tags)
logrus.Debugf("Avaliable tags for %s: %v", fullImageName, tags)
currentDigest, err1 := getDigestForImage(services[0].ImageInfo.Registry, services[0].ImageInfo.Image, services[0].ImageInfo.Tag, dockerHubClient, ghcrClient, commonClients)
latestDigest, err2 := getDigestForImage(services[0].ImageInfo.Registry, services[0].ImageInfo.Image, "latest", dockerHubClient, ghcrClient, commonClients)
if err != nil {
logrus.Errorf("Error getting tags for %s: %v", fullImageName, err)
// --- MODIFICATION: Suppress duplicate error messages ---
notificationKey := fullImageName + ":error_getting_tags"
sentNotificationsMutex.Lock()
if sentNotifications[notificationKey] {
sentNotificationsMutex.Unlock()
continue
}
sentNotifications[notificationKey] = true
sentNotificationsMutex.Unlock()
chatID, _ := strconv.ParseInt(appConfig.TelegramChatID, 10, 64)
notifier.SendTelegramMessage(bot, chatID, fmt.Sprintf("❌ Error getting tags for %s: %v", fullImageName, err))
// --- END MODIFICATION ---
continue
}
logrus.Debugf("Digest fallback for %s: currentDigest=%s (err=%v), latestDigest=%s (err=%v)",
fullImageName, currentDigest, err1, latestDigest, err2)
// Filter and sort tags according to SemVer and propose N latest
latestTags := versionutils.SortAndFilterTags(tags, appConfig.Checker.ProposedTagsCount, appConfig.Checker.IgnoreKeywords)
if len(latestTags) == 0 {
logrus.Warnf("⚠️ Can't check updates for %s. Current tag: %s, it is not use SemVer template. Available tags: %v",
fullImageName, currentImageTag, tags)
currentDigest, err1 := getDigestForImage(
services[0].ImageInfo.Registry,
services[0].ImageInfo.Image,
services[0].ImageInfo.Tag,
dockerHubClient,
ghcrClient,
commonClients, // Pass it here
)
latestDigest, err2 := getDigestForImage(
services[0].ImageInfo.Registry,
services[0].ImageInfo.Image,
"latest",
dockerHubClient,
ghcrClient,
commonClients, // And here
)
logrus.Debugf("Digest fallback for %s: currentDigest=%s (err=%v), latestDigest=%s (err=%v)",
fullImageName, currentDigest, err1, latestDigest, err2)
if err1 == nil && err2 == nil && currentDigest != "" && latestDigest != "" {
if currentDigest != latestDigest {
logrus.Warnf("🚨 Digest update for %s. Current tag: %s, 'latest' tag digest differs!", fullImageName, currentImageTag)
// --- MODIFICATION: Suppress duplicate digest update messages ---
notificationKey := fullImageName + ":digest_update"
sentNotificationsMutex.Lock()
if sentNotifications[notificationKey] {
sentNotificationsMutex.Unlock()
continue
}
sentNotifications[notificationKey] = true
sentNotificationsMutex.Unlock()
chatID, _ := strconv.ParseInt(appConfig.TelegramChatID, 10, 64)
notifier.SendTelegramMessage(bot, chatID, fmt.Sprintf("🚨 Digest update for %s. Current tag: %s, 'latest' tag digest differs!", fullImageName, currentImageTag))
// --- END MODIFICATION ---
} else {
logrus.Infof("%s is actual by digest. Current tag: %s", fullImageName, currentImageTag)
}
} else {
logrus.Errorf("Can't check updates for %s. No SemVer tags and no digest fallback.", fullImageName)
}
continue
}
// Check the latest tag is newer than the current tag
latestAvailable := latestTags[0]
// --- MODIFICATION: Try to normalize current tag before parsing ---
normalizedCurrentTag := versionutils.TryNormalizeTagForSemVer(currentImageTag)
currentVersion, err := semver.NewVersion(strings.TrimPrefix(normalizedCurrentTag, "v"))
// --- END MODIFICATION ---
if err != nil {
logrus.Warnf("⚠️ Can't check updates for %s. Current tag: %s, it is not use SemVer template. Available tags: %v",
fullImageName, currentImageTag, tags)
// --- MODIFICATION: Suppress duplicate warning messages ---
notificationKey := fullImageName + ":" + currentImageTag + ":non_semver_warning"
sentNotificationsMutex.Lock()
if sentNotifications[notificationKey] {
sentNotificationsMutex.Unlock()
continue
}
sentNotifications[notificationKey] = true
sentNotificationsMutex.Unlock()
chatID, _ := strconv.ParseInt(appConfig.TelegramChatID, 10, 64)
notifier.SendTelegramMessage(bot, chatID, fmt.Sprintf("⚠️ Can't check updates for %s. Current tag: %s, it is not use SemVer template.", fullImageName, currentImageTag))
// --- END MODIFICATION ---
continue
}
latestVersion, _ := semver.NewVersion(strings.TrimPrefix(latestAvailable, "v"))
if latestVersion.GreaterThan(currentVersion) {
for _, tag := range latestTags {
if sl.IsSkipped(fullImageName, tag) {
logrus.Infof("Version %s for %s is in the skip list. Skipping notification.", tag, fullImageName)
continue
}
// Don't notify if the tag is already in use
if tag == services[0].ImageInfo.Tag {
logrus.Infof("Tag %s for %s is already in use. Skipping notification.", tag, fullImageName)
continue
}
// --- NEW LOGIC: Check if notification already sent ---
notificationKey := fullImageName + ":" + tag
if err1 == nil && err2 == nil && currentDigest != "" && latestDigest != "" {
if currentDigest != latestDigest {
logrus.Warnf("🚨 Digest update for %s. Current tag: %s, 'latest' tag digest differs!", fullImageName, currentImageTag)
notificationKey := fullImageName + ":digest_update"
sentNotificationsMutex.Lock()
if sentNotifications[notificationKey] {
logrus.Infof("Notification for %s with tag %s already sent. Skipping duplicate.", fullImageName, tag)
sentNotificationsMutex.Unlock()
continue
}
sentNotifications[notificationKey] = true // Mark as sent
sentNotifications[notificationKey] = true
sentNotificationsMutex.Unlock()
// --- END NEW LOGIC ---
logrus.Infof("🚨 New update for %s. Current tag: %s, Available: %s", fullImageName, services[0].ImageInfo.Tag, tag)
// Generate unique ID for this update
// Using timestamp for simplicity; in production, consider a more robust method
// to avoid collisions.
id := fmt.Sprintf("%x", time.Now().UnixNano())
// Save update info in cache
cacheMutex.Lock()
updateCache[id] = UpdateInfo{
RepoPath: appConfig.Git.LocalRepoPath,
Services: services,
NewTag: tag,
}
cacheMutex.Unlock()
message := fmt.Sprintf(
"New update for `%s`. \n"+
"Current version: `%s`. \n"+
"Avaliable version: `%s`.",
fullImageName,
services[0].ImageInfo.Tag,
tag,
)
// Before send message — encode callback data
callbackData := encodeCallbackData("update_to", fullImageName, tag)
// If hash-mode, save to map
if strings.Contains(callbackData, ":h:") {
hash := strings.Split(callbackData, ":h:")[1]
callbackHashMap[hash] = [2]string{fullImageName, tag}
}
notifier.SendInteractiveMessage(bot, appConfig.TelegramChatID, message, []string{tag}, callbackData)
chatID, _ := strconv.ParseInt(appConfig.TelegramChatID, 10, 64)
notifier.SendTelegramMessage(bot, chatID, fmt.Sprintf("🚨 Digest update for %s. Current tag: %s, 'latest' tag digest differs!", fullImageName, currentImageTag))
} else {
logrus.Infof("%s is up to date by digest. Current tag: %s", fullImageName, currentImageTag)
}
// --- MODIFICATION: Clear warnings if an update is found ---
sentNotificationsMutex.Lock()
delete(sentNotifications, fullImageName+":"+currentImageTag+":non_semver_warning")
sentNotificationsMutex.Unlock()
// --- END MODIFICATION ---
} else {
logrus.Infof("✅ %s is actual. Current tag: %s", fullImageName, currentImageTag)
// --- NEW LOGIC: Clear any pending notifications for this image if it's now actual ---
sentNotificationsMutex.Lock()
for key := range sentNotifications {
if strings.HasPrefix(key, fullImageName+":") {
delete(sentNotifications, key)
}
}
sentNotificationsMutex.Unlock()
// --- END NEW LOGIC ---
logrus.Errorf("Can't check updates for %s. No SemVer tags and no digest fallback.", fullImageName)
}
continue
}
// Compare current tag against the latest available tag.
latestAvailable := latestTags[0]
normalizedCurrentTag := versionutils.TryNormalizeTagForSemVer(currentImageTag)
currentVersion, err := semver.NewVersion(strings.TrimPrefix(normalizedCurrentTag, "v"))
if err != nil {
logrus.Warnf("⚠️ Can't parse current tag for %s as SemVer. Tag: %s", fullImageName, currentImageTag)
notificationKey := fullImageName + ":" + currentImageTag + ":non_semver_warning"
sentNotificationsMutex.Lock()
if sentNotifications[notificationKey] {
sentNotificationsMutex.Unlock()
continue
}
sentNotifications[notificationKey] = true
sentNotificationsMutex.Unlock()
chatID, _ := strconv.ParseInt(appConfig.TelegramChatID, 10, 64)
notifier.SendTelegramMessage(bot, chatID, fmt.Sprintf("⚠️ Can't check updates for %s. Current tag: %s is not a SemVer tag.", fullImageName, currentImageTag))
continue
}
latestVersion, err := semver.NewVersion(strings.TrimPrefix(versionutils.TryNormalizeTagForSemVer(latestAvailable), "v"))
if err != nil || latestVersion == nil {
logrus.Warnf("⚠️ Can't parse latest tag %q for %s as SemVer, skipping.", latestAvailable, fullImageName)
continue
}
if latestVersion.GreaterThan(currentVersion) {
for _, tag := range latestTags {
if sl.IsSkipped(fullImageName, tag) {
logrus.Infof("Version %s for %s is in the skip list. Skipping notification.", tag, fullImageName)
continue
}
if tag == services[0].ImageInfo.Tag {
logrus.Infof("Tag %s for %s is already in use. Skipping notification.", tag, fullImageName)
continue
}
// Only notify for tags that are strictly newer than the current one.
tagVersion, err := semver.NewVersion(strings.TrimPrefix(versionutils.TryNormalizeTagForSemVer(tag), "v"))
if err != nil || tagVersion == nil || !tagVersion.GreaterThan(currentVersion) {
logrus.Debugf("Tag %s for %s is not newer than current %s, skipping.", tag, fullImageName, currentImageTag)
continue
}
notificationKey := fullImageName + ":" + tag
sentNotificationsMutex.Lock()
if sentNotifications[notificationKey] {
logrus.Infof("Notification for %s with tag %s already sent. Skipping duplicate.", fullImageName, tag)
sentNotificationsMutex.Unlock()
continue
}
sentNotifications[notificationKey] = true
sentNotificationsMutex.Unlock()
logrus.Infof("🚨 New update for %s. Current tag: %s, Available: %s", fullImageName, services[0].ImageInfo.Tag, tag)
id := fmt.Sprintf("%x", time.Now().UnixNano())
cacheMutex.Lock()
updateCache[id] = UpdateInfo{
RepoPath: appConfig.Git.LocalRepoPath,
Services: services,
NewTag: tag,
}
cacheMutex.Unlock()
message := fmt.Sprintf(
"New update for `%s`. \n"+
"Current version: `%s`. \n"+
"Available version: `%s`.",
fullImageName,
services[0].ImageInfo.Tag,
tag,
)
callbackData := encodeCallbackData("update_to", fullImageName, tag)
if strings.Contains(callbackData, ":h:") {
hash := strings.Split(callbackData, ":h:")[1]
callbackHashMap[hash] = [2]string{fullImageName, tag}
}
notifier.SendInteractiveMessage(bot, appConfig.TelegramChatID, message, []string{tag}, callbackData)
}
// Clear any stale non-semver warning now that we found valid tags.
sentNotificationsMutex.Lock()
delete(sentNotifications, fullImageName+":"+currentImageTag+":non_semver_warning")
sentNotificationsMutex.Unlock()
} else {
logrus.Infof("✅ %s is up to date. Current tag: %s", fullImageName, currentImageTag)
// Image is current — clear all cached notifications so they can re-fire
// if something changes later.
sentNotificationsMutex.Lock()
for key := range sentNotifications {
if strings.HasPrefix(key, fullImageName+":") {
delete(sentNotifications, key)
}
}
sentNotificationsMutex.Unlock()
}
}
logrus.Info("Image check completed.")
}
// Encode callback data
// If the length exceeds 64 characters, fallback to sha1 hash.
// encodeCallbackData encodes prefix + image + tag into a Telegram callback string.
// Falls back to a sha1 hash when the encoded value exceeds 64 bytes.
func encodeCallbackData(prefix, fullImageName, tag string) string {
raw := fullImageName + ":" + tag
enc := base64.RawURLEncoding.EncodeToString([]byte(raw))
@@ -565,12 +539,11 @@ func encodeCallbackData(prefix, fullImageName, tag string) string {
if len(data) <= 64 {
return data
}
// fallback на sha1
hash := sha1.Sum([]byte(raw))
return prefix + ":h:" + hex.EncodeToString(hash[:])
}
// Get image digest for given registry, image and tag
// getDigestForImage returns the manifest digest for a given registry/image/tag.
func getDigestForImage(registryName, image, tag string, dockerHubClient *registry.DockerHubClient, ghcrClient *registry.GhcrClient, commonClients map[string]*registry.CommonRegistryClient) (string, error) {
switch registryName {
case "docker.io":