Integration with Jenkins builds?

[crfrolik]

Which Renovate are you using?

We aren't using any yet. We're in the process of evaluating whether Renovate will work for us.

Which platform are you using?

GitHub Enterprise

Have you checked the logs? Don't forget to include them if relevant

N/A

What would you like to do?

We have several internal dependencies that get built by our Jenkins server.
Our build process downloads artifacts from each and unzips them into the workspace.

E.g.

https://our-jenkins-server/job/dependency1/job/dev/304/artifact/bundle.zip

I'm wondering if it's possible to integrate this with RenovateBot, since Jenkins does have a REST API for getting build number information.

[viceice]

Jenkins will send build status to GitHub (GitHub plugin should work for gh enterprise too).

Renovate will use build status read from GitHub.

I can send you a Jenkins pipeline for running renovate with schedule. You simply need to configure GitHub enterprise instead of bitbucket server, which I use.

[crfrolik]

I can send you a Jenkins pipeline for running renovate with schedule. You simply need to configure GitHub enterprise instead of bitbucket server, which I use.

That would be awesome, thank you.

[viceice]

pipeline {
    agent {
        docker { 
            image 'renovate/renovate:20.6.1-slim@sha256:94e4acb78e6562f37c06182869f80aec702dae419be208c94d83a691e1302073'
            args '-v /var/run/docker.sock:/var/run/docker.sock'
        }
    }

    environment {
        TZ = 'Europe/Berlin'
        LOG_LEVEL = 'debug'
        RENOVATE_PASSWORD = credentials('svc-ci-token')
        RENOVATE_TMPDIR = "${env.WORKSPACE}/.tmp"
        RENOVATE_DOCKER_USER = 'root'
        GITHUB_COM_TOKEN = credentials('gh-token')
        NPM_TOKEN = credentials('VO_NPM_TOKEN')
        LOG_LEVEL = 'debug'
        GIT_AUTHOR_NAME = 'Renovate Bot'
        GIT_AUTHOR_EMAIL = 'bot@visualon.de'
        GIT_COMMITTER_NAME = 'Renovate Bot'
        GIT_COMMITTER_EMAIL = 'bot@visualon.de'
    }

    parameters {
        string defaultValue: '', description: '', name: 'RENOVATE_ARGS', trim: true
    }

    options {
        disableConcurrentBuilds()
        timeout(time: 1, unit: 'HOURS')
        ansiColor('xterm')
        buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: env.BRANCH_NAME == 'master' ? '240' : '')
        office365ConnectorWebhooks([[name: 'teams/jenkins', notifyBackToNormal: true, notifyFailure: true, notifySuccess: false, notifyUnstable: true, url: env.TEAMS_HOOK ]])
    }
    triggers {
        // cron('H/30 6-17 * * 1-5')
        // cron('H/30 * * * *')
        // cron('H * * * *')
        cron(env.BRANCH_NAME == 'master' ? '@hourly' : '')
    }
    stages {
        stage('init') {
            steps {
                sh 'renovate --version'
                sh 'rm -f renovate.log'
            }
        }

        stage('test') {
            when { not { branch 'master' } }

            environment {
                RENOVATE_GIT_AUTHOR = 'Renovate Test Bot <bot@xxx.test>'
                GIT_AUTHOR_NAME = 'Renovate Test Bot'
                GIT_COMMITTER_NAME = 'Renovate Test Bot'
            }

            steps {
                sh 'renovate --dry-run --log-file renovate.log --log-file-level debug --autodiscover-filter \'test/*\''
            }
        }

        stage('renovate') {
            when { branch 'master' }
            steps {
                sh "renovate --log-file renovate.log --log-file-level debug ${params.RENOVATE_ARGS}"
            }
        }
    }
    post {
        always {
            archiveArtifacts allowEmptyArchive: true, artifacts: 'renovate.log'
            step([$class: 'Mailer', notifyEveryUnstableBuild: true, recipients: 'xxx@xxx.test', sendToIndividuals: true])
        }
    }
}

I have a config.js file in cwd, which conatains more specific renovate config for me.

The pipeline is run from bitbucket git repo and renovate runs on most of the git repos on same bitbucket server (selfupdating)

partial config for updating docker image in Jenkinsfile

...
"regexManagers": [
    {
      "fileMatch": ["^Jenkinsfile$"],
      "matchStrings": [
        "image '(?<depName>[a-z/-]+):(?<currentValue>[a-z0-9.-]+)@(?<currentDigest>sha256:[a-f0-9]+)'"
      ],
      "datasourceTemplate": "docker",
      "versioningTemplate": "docker"
    }
  ]

[viceice]

@crfrolik Can you please git some more detailed information, what kind of dependencies renovate should update?

[crfrolik]

@crfrolik Can you please git some more detailed information, what kind of dependencies renovate should update?

We have multiple kinds of dependencies:

1. JavaScript dependencies that come from yarn install; these actually live in our own private npm registry using verdaccio that we use via yarn config set registry
2. .NET dependencies installed via NuGet; once again, we have a private NuGet registry via ProGet
3. Python dependencies that come from pip
4. Custom dependencies that we download+unzip from Jenkins

I'm not too worried about the first three since I believe renovate can handle them.
The fourth one is what I'm concerned about; these deps are really binary applications that we install and use both in our build process and as part of our application.

Here's a small subset of the bash script we use that does this:

# Download the zip file from Jenkins
TEMP_DIR=$(mktemp -d)
ZIP_FILE="$TEMP_DIR/$ARCHIVE_NAME.zip"

ARCHIVE_URL="$JOB_URL/$BUILD_NUM/artifact/$ARCHIVE_NAME.zip"
wget -q $WGET_AUTH_PARAMS $ARCHIVE_URL -O $ZIP_FILE

# Extract and merge its artifacts folder
unzip -q $ZIP_FILE -d $TEMP_DIR
rsync -r "$TEMP_DIR/artifacts/" $OUT_DIR

Right now he have hard-coded BUILD_NUM="lastStableBuild" but that is a bad idea and we're looking to pin that to a specific number, and have Renovate update that number with pull requests.

Thanks for all the help so far.

[viceice]

Ok, so the update part is easy, as we can use a regex manager, but finding latest version needs a new Jenkins datasource.

So you can open a feature request on main repo for a Jenkins datasource. It would fetch all builds as version for a given job (job=depName, fullJobName=lookupName).

[crfrolik]

So you can open a feature request on main repo for a Jenkins datasource. It would fetch all builds as version for a given job (job=depName, fullJobName=lookupName).

I just searched and found this issue already entered, which I believe covers my use case:
renovatebot/renovate#6223
Unfortunately it hasn't been implemented yet.

Jenkins will send build status to GitHub (GitHub plugin should work for gh enterprise too).

This sounds promising; is it an alternative to waiting on the feature request to be implemented? What plugin are you referring to?

[viceice]

So you can open a feature request on main repo for a Jenkins datasource. It would fetch all builds as version for a given job (job=depName, fullJobName=lookupName).

I just searched and found this issue already entered, which I believe covers my use case:
renovatebot/renovate#6223
Unfortunately it hasn't been implemented yet.

I don't think we can implement this in near future, so we need help from community for this.

Jenkins will send build status to GitHub (GitHub plugin should work for gh enterprise too).

This sounds promising; is it an alternative to waiting on the feature request to be implemented? What plugin are you referring to?

https://resources.github.com/whitepapers/practical-guide-to-CI-with-Jenkins-and-GitHub/

[crfrolik]

https://resources.github.com/whitepapers/practical-guide-to-CI-with-Jenkins-and-GitHub/

Thank you, I will take a look at this. I think that this issue can be closed; I appreciate your help.