Local Renovate Testing

Purpose

To quickly iterate testing changes to renovate.json. As a reminder, renovate will only run against the development branch, so your changes must be merged to development before either the renovate-bot or your local test will be effective. Have your Repo1 personal access token, Registry1 credentials and GitHub personal access token (steps are below on how to create it) handy because it is needed by the renovate-bot to communicate with their respective API.

How

Create a personal access token in GitHub (assuming you have an existing account in GitHub, create an account if you do not have one)

To communicate with GitHub's GraphQL server, you'll need an OAuth token with the right scopes. Follow steps below to create your GitHub personal access token.

Log in your account.
In the upper-right corner of any page, click your profile photo, then click Settings.
In the left sidebar, click Developer settings.
In the left sidebar, click Personal access tokens.
Click Generate new token.
Give your token a descriptive name.
To give your token an expiration, select the Expiration drop-down menu, then click a default or use the calendar picker.

Select the scopes below, or permissions, to grant this token.

repo

read:packages

read:org

read:public_key

read:repo_hook

user

read:discussion

read:enterprise

read:gpg_key

Click Generate token and securely save it.

Set your environment variable

Edit your ~/.zshrc or ~/.bashrc. Add below code block and replace with their respective value. Save and reload your terminal session.

export REPO1_TOKEN=replace-with-your-repo1.dso.mil-personal-access-token
export REPO1_ENDPOINT=https://repo1.dso.mil/api/v4 #Do not change this
export GITHUB_TOKEN=replace-with-your-github-personal-access-token # This the token you created above
export REGISTRY1_USERNAME=replace-with-your-registry1.dso.mil-username
export REGISTRY1_TOKEN=replace-with-your-registry1.dso.mil-cli-secret
export REGISTRY1_ENDPOINT=registry1.dso.mil #Do not change this

Testing renovate configuration in Repo1

Execute renovate test

Copy and paste the code block below in your terminal and replace RENOVATE_REPOSITORIES value with your target repo (ie. dsop/opensource/kubernetes/1.25/kubectl) then hit enter/return key.

docker run --rm -ti \
    -e RENOVATE_PLATFORM=gitlab \
    -e RENOVATE_ENDPOINT=$REPO1_ENDPOINT \
    -e RENOVATE_TOKEN=$REPO1_TOKEN \
    -e RENOVATE_AUTODISCOVER=false \
    -e RENOVATE_REPOSITORIES=dsop/opensource/grafana/grafana \
    -e GITHUB_COM_TOKEN=$GITHUB_TOKEN \
    -e RENOVATE_HOST_RULES='[{"hostType":"docker","matchHost":"'"$REGISTRY1_ENDPOINT"'","username":"'"$REGISTRY1_USERNAME"'","password":"'"$REGISTRY1_TOKEN"'"}]' \
registry1.dso.mil/ironbank/container-hardening-tools/renovate/renovate:latest

Expected Output:

 INFO: Repository started (repository=dsop/opensource/grafana/grafana)
       "renovateVersion": "32.120.0"
 INFO: Dependency extraction complete (repository=dsop/opensource/grafana/grafana)
       "baseBranch": "development",
       "stats": {
         "managers": {
           "dockerfile": {"fileCount": 1, "depCount": 1},
           "regex": {"fileCount": 1, "depCount": 1},
           "ironbank": {"fileCount": 1, "depCount": 1}
         },
         "total": {"fileCount": 3, "depCount": 3}
       }
 INFO: Repository finished (repository=dsop/opensource/grafana/grafana)
       "durationMs": 7448

Execute renovate test with debug. Useful for troubleshooting.

Copy and paste the code block below in your terminal and replace RENOVATE_REPOSITORIES value with your target repo (ie. dsop/opensource/kubernetes/1.25/kubectl) then hit enter/return key.

docker run --rm -ti \
    -e RENOVATE_PLATFORM=gitlab \
    -e RENOVATE_ENDPOINT=$REPO1_ENDPOINT \
    -e RENOVATE_TOKEN=$REPO1_TOKEN \
    -e RENOVATE_AUTODISCOVER=false \
    -e RENOVATE_REPOSITORIES=dsop/opensource/grafana/grafana \
    -e GITHUB_COM_TOKEN=$GITHUB_TOKEN \
    -e RENOVATE_HOST_RULES='[{"hostType":"docker","matchHost":"'"$REGISTRY1_ENDPOINT"'","username":"'"$REGISTRY1_USERNAME"'","password":"'"$REGISTRY1_TOKEN"'"}]' \
    -e LOG_LEVEL=debug \
registry1.dso.mil/ironbank/container-hardening-tools/renovate/renovate:latest

If you did a renovate test on a repo where the renovate-bot successfully created a pull request (PR) then you manually closed the PR. In the next test run, renovate-bot will skip creating a new PR since it can detect the previously closed PR that it contains the same title. In order to override it, add -e RENOVATE_RECREATE_CLOSED=true flag in the code block above.

Testing in a local environment using a Gitea instance.

Purpose

Testing renovate.json locally will save you time from pushing changes to development branch in Repo1, plus you will need someone to review/approve your changes. This will give you freedom to troubleshoot renovate.json with no worries of making changes in Repo1.

You will have the ability to test/troubleshoot a repo's renovate.json whenever you need and re-use the test personal access tokens, unless you delete /gitea and /mysql directories after your first deployment of the Gitea instance.

To have an effective renovate local test, have your Repo1 personal access token, Registry1 credentials and GitHub personal access token.

Setup Gitea instance.

Download docker-compose.yml file in preferred local directory.
Change directory to where you downloaded the file.

Create gitea and mysql directories.

~/repos/playground/test-renovate
$ mkdir gitea mysql

Deploy Gitea through docker-compose.

~/repos/playground/test-renovate
$ docker-compose up -d
Creating network "test-renovate_gitea" with the default driver
Creating test-renovate_db_1 ... done
Creating gitea              ... done

Go to http://localhost:3000.
Click Install Gitea on the bottom of your screen.
Refresh http://localhost:3000.
Click Register (top right of your screen).
Fill out registration.
```
Username: renovate
Email Address: 123@example.com
Password: password
Re-Type Password: password
```
- Click Register Account.
- Securely save newly registered account information. You will need the account information during the renovate test.
Create Gitea personal access token
- Click your profile on the top right of your scree then select Settings.
- Click Applications tab.
- Enter name under Token Name.
- Click Generate Token.
- Securely save it. You will need this token during the renovate test.
Create repo you want to test.
- Click + on top right.
- Click + New Repository.
- Enter repository name.
- Click Create Repository on the bottom of your screen.
- Clone the newly gitea repo locally.
Clone repo1 target repo.
- Make your necessary changes in the repo. (I like to downgrade the tag version of the container and respective resource to see if renovate can pickup changes from their respective datasource.)
- Make sure you change baseBranches value into main or 'master' in your renovate.json.
- Copy the files into the gitea repo you cloned.
Push your local gitea repo updates to the Gitea instance
- Add/commit/push your changes to Gitea instance.
```
cd /path/of/your/local/gitea/repo
git add .
git commit -m "renovate test"
git push
```

Initiate renovate test. Make sure to update the RENOVATE_TOKEN with the Gitea token you created above then run the code block below

docker run --rm -ti \
--network=host \
--platform=linux/x86_64 \
 -e RENOVATE_PLATFORM=gitea \
 -e RENOVATE_GIT_AUTHOR="renovate <123@example.com>" \
 -e RENOVATE_ENDPOINT="http://localhost:3000/api/v1/" \
 -e GITHUB_COM_TOKEN=$GITHUB_TOKEN \
 -e RENOVATE_TOKEN="replace-with-gitea-token" \
 -e RENOVATE_HOST_RULES='[{"hostType":"docker","matchHost":"'"$REGISTRY1_ENDPOINT"'","username":"'"$REGISTRY1_USERNAME"'",  "password":"'"$REGISTRY1_TOKEN"'"}]' \
 -e RENOVATE_AUTODISCOVER='true' \
 -e RENOVATE_RECREATE_CLOSED=true \
registry1.dso.mil/ironbank/container-hardening-tools/renovate/renovate:latest

If renovate test is successful, you should see that renovate-bot created a pull request and branch like the output below. If it fails/didnt see expected output/want to see debug information, you can add -e LOG_LEVEL=debug in the code block above.

    INFO: Autodiscovered repositories
          "length": 1,
          "repositories": ["renovate/rabbitmq310"]
    INFO: Repository started (repository=renovate/rabbitmq310)
          "renovateVersion": "32.120.0"
    INFO: Dependency extraction complete (repository=renovate/rabbitmq310)
          "baseBranch": "main",
          "stats": {
            "managers": {
              "dockerfile": {"fileCount": 1, "depCount": 2},
              "regex": {"fileCount": 1, "depCount": 2},
              "ironbank": {"fileCount": 1, "depCount": 3}
            },
            "total": {"fileCount": 3, "depCount": 7}
          }
    INFO: Branch created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-digest)
          "commitSha": "7039bec94c6007d9a569876a2a6096dfd13098f5"
    INFO: PR created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-digest)
          "pr": 11,
          "prTitle": "Update rabbitmq digest to 5d5de39"
    INFO: Branch created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-3.10.x)
          "commitSha": "7761be74811fa6da9b2a3f8766e53e969d03dcac"
    INFO: PR created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-3.10.x)
          "pr": 12,
          "prTitle": "Update dependency rabbitmq to v3.10.10"
    INFO: Repository finished (repository=renovate/rabbitmq310)
          "durationMs": 20416

12. Shutdown Gitea instance when you are done testing.

docker-compose down

Issue Grouping

It can sometimes be helpful to group all package updates into a single branch and merge request. To do so you must update the renovate.json to include a group preset like below:

{
  "assignees": [
    "@username"
  ],
  "baseBranches": [
    "development"
  ],
  "automerge": true,
  "gitLabAutomerge": true,
  "groupName": "all dependencies",
  "separateMajorMinor": false,
  "groupSlug": "all",
  "packageRules": [
    {
      "matchPackagePatterns": [
        "*"
      ],
      "groupName": "all dependencies",
      "groupSlug": "all",
      "datasources": [
        "docker"
      ],
      "packageNames": [
        "registry1.dso.mil/ironbank/opensource/python/python39"
      ]
    }
  ],
  "regexManagers": [
    {
      "fileMatch": [
        "^Dockerfile$"
      ],
      "matchStrings": [
        "BASE_TAG=(?<currentValue>.*?)"
      ],
      "depNameTemplate": "registry1.dso.mil/ironbank/opensource/python/python39",
      "datasourceTemplate": "docker"
    },
    {
      "fileMatch": [
        "^hardening_manifest.yaml$"
      ],
      "matchStrings": [
        "BASE_TAG:\\s+\"(?<currentValue>.*?)\""
      ],
      "depNameTemplate": "registry1.dso.mil/ironbank/opensource/python/python39",
      "datasourceTemplate": "docker"
    }
  ]
}