Creating Recurring Gitlab Issues

Ben Tasker

2024-09-30 15:55

I've long used project management software to track and manage tasks and chores at home. Although this has not always been a popular choice, I spend a chunk of my working day staring at issue trackers, so it's never really made sense to track things any differently at home.

I was using JIRA when this habit first formed, but I migrated to Gitlab after Atlassian decided to effectively end their on-prem offering (at the time, they priced themselves out of the market, ahead of dropping Server completely).

Although I'm generally happy with Gitlab, one feature that it lacks is the ability to create recurring issues in order to track tasks which need repeating periodically (for example, updating the base image used when building new container images).

Home has it's own set of examples, with "check water softener salt" being a good use-case: it's a task that generally only needs doing every now and then and so can easily be forgotten. Or, of course, there's also the annual oiling of the hinges.

I had been relying on calendar entries to track recurring needs, but it was a little jarring to have to track those in a different application.

A few weeks ago, I got around to addressing this by building a small container image which can parse a configuration file and raise tickets in Gitlab. This post details how to use that solution, in case others find it useful before Gitlab too thoroughly ruin their product (and the planet) with AI (snark, moi?).

Setting up Access

The script will need to be able to authenticate with Gitlab.

I chose to create a dedicated service account, though you could also use your own.

Once you're logged in, you need to create a Personal Access Token:

Click your Avatar
Choose Preferences
Choose Access Tokens
Set a name
Grant api scope
Click Create personal access token
Take a note of the token

If you've created a service account, you'll also need to ensure that it's added into relevant projects. If you want the script to be able to label and assign the issues that it creates, it'll need Developer permissions (without those, though, the bare tickets will still be created).

Although it'll work without, the system also works best if you've setup SMTP access in Gitlab so that you receive email notifications when issues get assigned to you.

Creating Configuration

Tickets are defined in a YAML configuration file:

# Labels specified here will be added to all created issues
# These are optional
labels: ["RecurringIssue"]

# Define the tickets to create
tickets:
  - title: "Check Softener Salt"
    # Should this ticket be created?
    active: true
    # Scheduling for the ticket 
    schedule:
        nth:
            n: 3
            weekday: Sat
    # Ticket description. This can also be a multiline
    # markdown string
    description: "Check whether the water softener needs more salt adding"
    # namespace + project to create ticket in
    project: jira-projects/HOME
    # User to assign the ticket to
    assignee: btasker
    # Optional - set a due date on the ticket
    due_in_days: 7
    # Labels to add to the ticket
    labels: ["Task","kitchen"]

  # Onto the next ticket
  - title: "Switch to Winter Charge Schedule"
    active: true
    schedule:
        day: 28
        month: 9
    description: |
       Switch the battery to the winter charge schedule

       See [Battery Charge Schedule](https://shrt/battery) for settings
    project: misc/solar
    assignee: btasker
    labels: ["Task","Battery","Configuration"]

Labels don't need to have been pre-created in Gitlab - if they don't exist they'll be created. I don't know that I love that, but it's the way that the API works rather than a conscious design choice on my part.

One of the advantages of setting due_in_days is that it'll add a due date to the created ticket. As that date approaches, Gitlab will start to nag you to actually get it done in time.

It's also possible to move descriptions into dedicated files allowing for much longer & more complex tickets to be raised.

Scheduling Syntax

The schedule attribute can use a range of different mechanisms. More than one mechanism can be used at a time, with the first match winning.

Named Day

It's possible to specify a day of the month:

schedule:
  day: 4
  month: Sep

The month attribute is optional (if omitted then every month matches) and can be a (localised) short-name or a number.

Multiple days or months can be provided by separating them with a forward slash

schedule:
  day: 4/9/21

Every `$d`

It's also possible to use every

schedule:
  every: Sun

This accepts weekday names or numbers (0 is Monday, 6 Sunday).

Weekday names use the configured locale, so in fr_FR you might set

schedule:
  every: Dim

every also accepts the value run (which specifies that it should trigger everytime the script is run - this is only generally useful for debugging).

`n`th Day Scheduling

The script can also use nth day scheduling, allowing tickets to be scheduled based on a more abstract position in the month.

For example, to raise a ticket on the second Sunday of each month:

schedule:
  nth:
    n: 2
    weekday: Sun

Note: It's not currently possible to specify which month, so you can't say "the second Saturday of July".

Running

The script is designed to be run using the container image, which can be used with docker, podman or Kubernetes.

The advantage of using Kubernetes is that you can create a CronJob. The disadvantage is that, err, you're using Kubernetes - unless you already have a cluster handy, it's likely to be massive overkill.

If you're feeling really posh, you could also use Gitlab's CI to create a scheduled pipeline which runs the container once a day.

Docker

Docker invocation is simple, you should pass your Gitlab token through along with the URL of the Gitlab server and map the config file through

docker run --rm \
-e GITLAB_TOKEN="<my token>" \
-e GITLAB_SERVER="https://gitlab.example.com" \
-v $PWD/examples/example_config.yml:/config.yml \
ghcr.io/bentasker/gitlab_recurring_issue:0.1

You'll need to handle scheduling yourself - this might just be a cron job which runs the container once daily.

podman, of course, uses an almost identical command.

Kubernetes

Kubernetes has secret storage available, so rather than adding a privileged credential directly into the YAML, you'll first want to create a secret

kubectl create secret generic gitlab-auth \
--from-literal=token=<my token>

You can also write your configuration into a ConfigMap

---

apiVersion: v1
kind: ConfigMap
metadata:
  name: gitlab-recurring-issues
data:
  config.yml: |
    # Labels specified here will be added to all created issues
    labels: ["RecurringIssue"]

    # Define the tickets to create
    tickets:
      - title: "Check Softener Salt"
        active: true
        schedule:
            nth:
                n: 3
                weekday: Sat
        description: "Check whether the water softener needs more salt adding"
        project: jira-projects/HOME
        assignee: btasker
        due_in_days: 7
        labels: ["Task","kitchen"]
      - title: "Switch to Winter Charge Schedule"
        active: true
        due_in_days: 7
        schedule:
            day: 28
            month: 9
        description: |
           Switch the battery to the winter charge schedule

           See [Battery Charge Schedule](https://shrt/battery) for settings
        project: misc/solar
        assignee: btasker
        labels: ["Task","Battery","Configuration"]

Then, define the CronJob itself

---

apiVersion: batch/v1
kind: CronJob
metadata:
  name: gitlab-recurring-issues
spec:
  schedule: "30 5 * * *"
  failedJobsHistoryLimit: 5
  successfulJobsHistoryLimit: 5
  jobTemplate:
    spec:
        template:
            spec:
                restartPolicy: Never
                containers:
                - name: gitlab-recurring
                  image: ghcr.io/bentasker/gitlab_recurring_issue:0.1
                  imagePullPolicy: IfNotPresent
                  volumeMounts:
                  - mountPath: /config.yml
                    name: configfile
                    subPath: config.yml
                  env:
                  - name: DRY_RUN
                    value: "false"
                  - name: GITLAB_URL
                    value: "https://gitlab.example.com                    
                  - name: GITLAB_TOKEN
                    valueFrom: 
                        secretKeyRef:
                            name: "gitlab-auth"
                            key: token
                volumes:
                - name: configfile
                  configMap:
                    name: gitlab-recurring-issues

Apply the config

kubectl apply -f gitlab-recurring-issue.yml

and the job should fire at the scheduled time (0530 daily).

Debugging

You probably don't want to have to wait for a run to fire to check that everything's OK.

The script supports a couple of environment variables which can be used to aid testing and debugging

DRY_RUN: If set to true, the system will print to stdout rather than creating tickets
FORCE: If set to true, date validation will be overridden, forcing all tickets to trigger

docker run --rm \
-e GITLAB_TOKEN="<my token>" \
-e GITLAB_SERVER="https://gitlab.example.com" \
-e DRY_RUN=true \
-e FORCE=true \
-v $PWD/config.yml:/config.yml \
ghcr.io/bentasker/gitlab_recurring_issue:0.1

I added the global label configuration item to make it easier to find and fix mistakes when debugging: tidying up after you've accidentally run with FORCE=true DRY_RUN=false can be a bit of a pain otherwise.

Having at least once global label makes this much easier because you can simply log into Gitlab and search for all issues with that label.

Conclusion

I now have a system which can quite reliably raise tickets for recurring tasks, whether that's switching the solar between Winter and Summer schedules or booking the boiler's annual service.

It's by no means a perfect solution: the script being stateless means that it's not possible to set a schedule like "every 3 weeks". nth day scheduling generally gets things close enough though.

I have found that there are tasks that I want reminding about, but don't really consider important enough to have them filling up the project's issue tracker (completing the online weekly food shop being one such example).

I also found that there were some projects that I didn't particularly want to give the service account privileged access to.

I addressed both of these by creating a new "Generic tasks" project for issues to be raised against. Mundane tasks remain in that project, whilst those relating to slightly more sensitive projects can then be manually moved.

Links: