Local Renovate Testing
Purpose
To quickly iterate testing changes to renovate.json. As a reminder, renovate will only run against the development branch, so your changes must be merged to development before either the renovate-bot or your local test will be effective. Have your Repo1 personal access token, Registry1 credentials and GitHub personal access token (steps are below on how to create it) handy because it is needed by the renovate-bot to communicate with their respective API.
How
Create a personal access token in GitHub (assuming you have an existing account in GitHub, create an account if you do not have one)
To communicate with GitHub's GraphQL server, you'll need an OAuth token with the right scopes. Follow steps below to create your GitHub personal access token.
- Log in your account.
- In the upper-right corner of any page, click your profile photo, then click Settings.
- In the left sidebar, click Developer settings.
- In the left sidebar, click Personal access tokens.
- Click Generate new token.
- Give your token a descriptive name.
- To give your token an expiration, select the Expiration drop-down menu, then click a default or use the calendar picker.
-
Select the scopes below, or permissions, to grant this token.
repo
read:packages read:org read:public_key read:repo_hook
user
read:discussion read:enterprise read:gpg_key
-
Click Generate token and securely save it.
Set your environment variable
Edit your ~/.zshrc or ~/.bashrc. Add below code block and replace with their respective value. Save and reload your terminal session.
export REPO1_TOKEN=replace-with-your-repo1.dso.mil-personal-access-token
export REPO1_ENDPOINT=https://repo1.dso.mil/api/v4 #Do not change this
export GITHUB_TOKEN=replace-with-your-github-personal-access-token # This the token you created above
export REGISTRY1_USERNAME=replace-with-your-registry1.dso.mil-username
export REGISTRY1_TOKEN=replace-with-your-registry1.dso.mil-cli-secret
export REGISTRY1_ENDPOINT=registry1.dso.mil #Do not change this
Testing renovate configuration in Repo1
Execute renovate test
Copy and paste the code block below in your terminal and replace RENOVATE_REPOSITORIES value with your target repo (ie. dsop/opensource/kubernetes/1.25/kubectl) then hit enter/return key.
docker run --rm -ti \
-e RENOVATE_PLATFORM=gitlab \
-e RENOVATE_ENDPOINT=$REPO1_ENDPOINT \
-e RENOVATE_TOKEN=$REPO1_TOKEN \
-e RENOVATE_AUTODISCOVER=false \
-e RENOVATE_REPOSITORIES=dsop/opensource/grafana/grafana \
-e GITHUB_COM_TOKEN=$GITHUB_TOKEN \
-e RENOVATE_HOST_RULES='[{"hostType":"docker","matchHost":"'"$REGISTRY1_ENDPOINT"'","username":"'"$REGISTRY1_USERNAME"'","password":"'"$REGISTRY1_TOKEN"'"}]' \
registry1.dso.mil/ironbank/container-hardening-tools/renovate/renovate:latest
Expected Output:
INFO: Repository started (repository=dsop/opensource/grafana/grafana)
"renovateVersion": "32.120.0"
INFO: Dependency extraction complete (repository=dsop/opensource/grafana/grafana)
"baseBranch": "development",
"stats": {
"managers": {
"dockerfile": {"fileCount": 1, "depCount": 1},
"regex": {"fileCount": 1, "depCount": 1},
"ironbank": {"fileCount": 1, "depCount": 1}
},
"total": {"fileCount": 3, "depCount": 3}
}
INFO: Repository finished (repository=dsop/opensource/grafana/grafana)
"durationMs": 7448
Execute renovate test with debug. Useful for troubleshooting.
Copy and paste the code block below in your terminal and replace RENOVATE_REPOSITORIES value with your target repo (ie. dsop/opensource/kubernetes/1.25/kubectl) then hit enter/return key.
docker run --rm -ti \
-e RENOVATE_PLATFORM=gitlab \
-e RENOVATE_ENDPOINT=$REPO1_ENDPOINT \
-e RENOVATE_TOKEN=$REPO1_TOKEN \
-e RENOVATE_AUTODISCOVER=false \
-e RENOVATE_REPOSITORIES=dsop/opensource/grafana/grafana \
-e GITHUB_COM_TOKEN=$GITHUB_TOKEN \
-e RENOVATE_HOST_RULES='[{"hostType":"docker","matchHost":"'"$REGISTRY1_ENDPOINT"'","username":"'"$REGISTRY1_USERNAME"'","password":"'"$REGISTRY1_TOKEN"'"}]' \
-e LOG_LEVEL=debug \
registry1.dso.mil/ironbank/container-hardening-tools/renovate/renovate:latest
If you did a renovate test on a repo where the renovate-bot successfully created a pull request (PR) then you manually closed the PR. In the next test run, renovate-bot will skip creating a new PR since it can detect the previously closed PR that it contains the same title. In order to override it, add -e RENOVATE_RECREATE_CLOSED=true
flag in the code block above.
Testing in a local environment using a Gitea instance.
Purpose
Testing renovate.json locally will save you time from pushing changes to development branch in Repo1, plus you will need someone to review/approve your changes. This will give you freedom to troubleshoot renovate.json with no worries of making changes in Repo1.
You will have the ability to test/troubleshoot a repo's renovate.json whenever you need and re-use the test personal access tokens, unless you delete /gitea and /mysql directories after your first deployment of the Gitea instance.
To have an effective renovate local test, have your Repo1 personal access token, Registry1 credentials and GitHub personal access token.
-
Setup Gitea instance.
- Download docker-compose.yml file in preferred local directory.
- Change directory to where you downloaded the file.
- Create gitea and mysql directories.
~/repos/playground/test-renovate $ mkdir gitea mysql
- Deploy Gitea through docker-compose.
~/repos/playground/test-renovate $ docker-compose up -d Creating network "test-renovate_gitea" with the default driver Creating test-renovate_db_1 ... done Creating gitea ... done
-
Go to http://localhost:3000.
- Click
Install Gitea
on the bottom of your screen. - Refresh http://localhost:3000.
- Click
Register
(top right of your screen). - Fill out registration.
Username: renovate Email Address: 123@example.com Password: password Re-Type Password: password
- Click
Register Account
. - Securely save newly registered account information. You will need the account information during the renovate test.
- Click
- Create Gitea personal access token
- Click your profile on the top right of your scree then select
Settings
. - Click
Applications
tab. - Enter name under
Token Name
. - Click
Generate Token
. - Securely save it. You will need this token during the renovate test.
- Click your profile on the top right of your scree then select
- Create repo you want to test.
- Click
+
on top right. - Click
+ New Repository
. - Enter repository name.
- Click
Create Repository
on the bottom of your screen. - Clone the newly gitea repo locally.
- Click
-
Clone repo1 target repo.
- Make your necessary changes in the repo. (I like to downgrade the tag version of the container and respective resource to see if renovate can pickup changes from their respective datasource.)
- Make sure you change
baseBranches
value intomain
or 'master' in your renovate.json. - Copy the files into the gitea repo you cloned.
-
Push your local gitea repo updates to the Gitea instance
- Add/commit/push your changes to Gitea instance.
cd /path/of/your/local/gitea/repo git add . git commit -m "renovate test" git push
- Add/commit/push your changes to Gitea instance.
-
Initiate renovate test. Make sure to update the RENOVATE_TOKEN with the Gitea token you created above then run the code block below
docker run --rm -ti \ --network=host \ --platform=linux/x86_64 \ -e RENOVATE_PLATFORM=gitea \ -e RENOVATE_GIT_AUTHOR="renovate <123@example.com>" \ -e RENOVATE_ENDPOINT="http://localhost:3000/api/v1/" \ -e GITHUB_COM_TOKEN=$GITHUB_TOKEN \ -e RENOVATE_TOKEN="replace-with-gitea-token" \ -e RENOVATE_HOST_RULES='[{"hostType":"docker","matchHost":"'"$REGISTRY1_ENDPOINT"'","username":"'"$REGISTRY1_USERNAME"'", "password":"'"$REGISTRY1_TOKEN"'"}]' \ -e RENOVATE_AUTODISCOVER='true' \ -e RENOVATE_RECREATE_CLOSED=true \ registry1.dso.mil/ironbank/container-hardening-tools/renovate/renovate:latest
If renovate test is successful, you should see that renovate-bot created a pull request and branch like the output below. If it fails/didnt see expected output/want to see debug information, you can add
-e LOG_LEVEL=debug
in the code block above.12. Shutdown Gitea instance when you are done testing.INFO: Autodiscovered repositories "length": 1, "repositories": ["renovate/rabbitmq310"] INFO: Repository started (repository=renovate/rabbitmq310) "renovateVersion": "32.120.0" INFO: Dependency extraction complete (repository=renovate/rabbitmq310) "baseBranch": "main", "stats": { "managers": { "dockerfile": {"fileCount": 1, "depCount": 2}, "regex": {"fileCount": 1, "depCount": 2}, "ironbank": {"fileCount": 1, "depCount": 3} }, "total": {"fileCount": 3, "depCount": 7} } INFO: Branch created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-digest) "commitSha": "7039bec94c6007d9a569876a2a6096dfd13098f5" INFO: PR created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-digest) "pr": 11, "prTitle": "Update rabbitmq digest to 5d5de39" INFO: Branch created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-3.10.x) "commitSha": "7761be74811fa6da9b2a3f8766e53e969d03dcac" INFO: PR created (repository=renovate/rabbitmq310, branch=renovate/rabbitmq-3.10.x) "pr": 12, "prTitle": "Update dependency rabbitmq to v3.10.10" INFO: Repository finished (repository=renovate/rabbitmq310) "durationMs": 20416
docker-compose down
Issue Grouping
It can sometimes be helpful to group all package updates into a single branch and merge request. To do so you must update the renovate.json
to include a group preset like below:
{
"assignees": [
"@username"
],
"baseBranches": [
"development"
],
"automerge": true,
"gitLabAutomerge": true,
"groupName": "all dependencies",
"separateMajorMinor": false,
"groupSlug": "all",
"packageRules": [
{
"matchPackagePatterns": [
"*"
],
"groupName": "all dependencies",
"groupSlug": "all",
"datasources": [
"docker"
],
"packageNames": [
"registry1.dso.mil/ironbank/opensource/python/python39"
]
}
],
"regexManagers": [
{
"fileMatch": [
"^Dockerfile$"
],
"matchStrings": [
"BASE_TAG=(?<currentValue>.*?)"
],
"depNameTemplate": "registry1.dso.mil/ironbank/opensource/python/python39",
"datasourceTemplate": "docker"
},
{
"fileMatch": [
"^hardening_manifest.yaml$"
],
"matchStrings": [
"BASE_TAG:\\s+\"(?<currentValue>.*?)\""
],
"depNameTemplate": "registry1.dso.mil/ironbank/opensource/python/python39",
"datasourceTemplate": "docker"
}
]
}