NAME

veda-new-jenkins-job - Create job(s) for Jenkins CI

SYNOPSIS

veda new-jenkins-job [-o name-desc|--output name-desc|--create|--update] [--skill interp] [--auto-trigger] [--job-name name-desc] [--git-branch-spec branch-spec] [--git-remote name] [--cli-jar file] [--url url] pkg-name...

DESCRIPTION

Create job definition(s) for running the tests of pkg-name..., which must be managed using the Git source control system, in a Jenkins CI instance.

In addition to the basic CI server, the job definitions created by this command make use of the following Jenkins plugins:

Git Plugin: required; this plugin is used for source code checkouts, both of the package under test and its dependencies;
xUnit Plugin: recommended; parses/integrate test results into Jenkins' job interface, providing statistics and trend graphs.

Command new-jenkins-job can either run in standalone mode, writing XML streams, or be directed to create/update jobs on a running server.

If none of -o, --output, --create or --update are used, or when name-desc resolves to -, the job definition is written to standard output.

OPTIONS

-o name-desc, --output name-desc

Specify that XML job definition(s) are to be written to file(s) named by name-desc; cf. NAME DESCRIPTORS below.

Note that new-jenkins-job remains a purely local operation when -o/--output is used; no Jenkins instance will be contacted.

Defaults to -, i.e. standard output;

--create, --update

Create/update the job definition(s) on a running Jenkins CI instance. May require --cli-jar and/or --url. See also EXAMPLES section below.

--skill interp

Tell Jenkins to use interpreter interp when running the tests. The chosen binary/wrapper script must be runnable without any command-line parameters, be able to execute without a GUI, and evaluate SKILL code (including inScheme forms) provided on its standard input.

Defaults to the bare-bones skill interpreter; please cf. VEDA_SKILL_BIN in veda help for more information on how interpreters are located.

--auto-trigger

Creates automatic build triggers by inverting @requires dependencies. Given two packages foo and bar, where the latter requires the former:

$ grep @requires pkg/foo/foo.ils pkg/bar/bar.ils
pkg/bar/bar.ils:;; @requires foo

The following invocation of new-jenkins-job creates two definitions:

$ veda new-jenkins-job foo bar --auto-trigger -o %s.xml
    Create foo.xml
    Create bar.xml

with foo.xml referencing bar as a child project:

$ grep childProjects *.xml
foo.xml:      <childProjects>bar</childProjects>

This maps to Post-build Actions/Build other projects in Jenkins' job configuration panel.

--job-name name-desc

Specify the naming convention for jobs; cf. NAME DESCRIPTORS below for a description of name-desc. Defaults to pkg-name, which is equivalent to %s;

--git-branch-spec branch-spec

Specify the branches to be watched by Jenkins.

The syntax is of the form: REPOSITORYNAME/BRANCH. In addition, BRANCH is recognized as a shorthand of */BRANCH, * is recognized as a wildcard, and ** is recognized as wildcard that includes the separator /. Therefore, origin/branches* would match origin/branches-foo but not origin/branches/foo, while origin/branches** would match both origin/branches-foo and origin/branches/foo.

Defaults to master;

--git-remote name

Specify that the job must be setup for watching remote name of the package's repository, rather than monitoring the latter directly. I.e., given a Git-managed package foo in /my/prj/dev/pkg/ such that:

$ cd /my/prj/dev/pkg/foo/
foo$ git config remote.stage.url
/my/prj/stage/foo.git

Without this option, /my/prj/dev/pkg/foo/.git would be watched by Jenkins. With --git-remote stage, /my/prj/stage/foo.git would be monitored instead;

--cli-jar file

Path to jenkins-cli.jar, Jenkins' command-line client.

Please refer to Jenkins CLI for more information on how to retrieve the file from the running server.

Defaults to $VEDA_JENKINS_CLI_JAR, or an error if the latter is not set and --create or --update is used;

--url url

URL of the running Jenkins server.

Given jenkins-cli.jar (cf. --cli-jar option) and a running server at http://localhost:8080, the following:

$ java -jar jenkins-cli.jar -s http://localhost:8080 help

can be used to test the connection. The corresponding new-jenkins-job options are:

--cli-jar jenkins-cli.jar --url http://localhost:8080

Defaults to $JENKINS_URL, or an error if the latter is not set and --create or --update is used;

NAME DESCRIPTORS

Name descriptors (denoted as name-desc above) describe how to generate series of names, one per pkg-name.

This discussion use output file name descriptors as an example, but keep in mind that Jenkins job names are generated using the same mechanism.

The basic version, which can only be used with a single pkg-name, is a simple string without any % character:

$ veda new-jenkins-job foo -o foo.xml
    Create foo.xml

This breaks down with multiple packages, as each job definition requires its own file:

$ veda new-jenkins-job foo bar -o foo.xml
*Error*: new-jenkins-job: Multiple packages require
-o/--output with a %s specifier.

A single %s causes the string to be used as a generator, where %s is a placeholder for the package name:

$ veda new-jenkins-job foo bar -o ../%s.xml
    Create ../foo.xml
    Create ../bar.xml

As with the printf family of functions, a literal % can be included in the result by doubling it:

$ veda new-jenkins-job foo bar -o %s-100%%.xml
    Create foo-100%.xml
    Create bar-100%.xml

Note that it is an error to include multiple %s, or to use any other combination of non-doubled % characters.

ENVIRONMENT

The behavior of this command is affected by the following variables:

VEDA_JENKINS_CLI_JAR: Default for --cli-jar option above;
JENKINS_URL: Default for --url option above.

These variables can be set from the shell or via one of the config files (cf. veda help, section ENVIRONMENT).

EXAMPLES

Let's start by creating a workarea with two packages foo and bar, the latter requiring the former (cf. veda help new-workarea and veda help new-package for details):

$ veda new-workarea my-project
    Create my-project
    Create my-project/.cdsenv
    Create my-project/cds.lib
    Create my-project/.cdsinit
    Create my-project/pkg/README
$ cd my-project/
my-project$ veda new-package foo --language scheme
    Create pkg/foo
    Create pkg/foo/foo.ils
    Create pkg/foo/foo-test.ils
my-project$ veda new-package bar --language scheme
    Create pkg/bar
    Create pkg/bar/bar.ils
    Create pkg/bar/bar-test.ils
my-project$ sed -e '2 s/^/;; @requires foo\n/' pkg/bar/bar.ils > tmp
my-project$ mv tmp pkg/bar/bar.ils

We turn these new packages into local Git repositories:

my-project$ git init pkg/foo/
Initialized empty Git repository in my-project/pkg/foo/.git/
my-project$ git init pkg/bar/
Initialized empty Git repository in my-project/pkg/bar/.git/

then use new-jenkins-job to create a standalone job definitions files, which we can examine at our leisure:

my-project$ veda new-jenkins-job foo bar -o testing-%s.xml
    Create testing-foo.xml
    Create testing-bar.xml

Setting up Jenkins CI

In our example, the server is to be run under user my-project-ci, on port 1920 of ci.acme.corp. Let's first log into the target machine:

$ ssh -l my-project-ci ci.acme.corp

fetch the latest version of Jenkins CI:

ci.acme.corp$ wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war
...
Saving to: 'jenkins.war'

and start it:

ci.acme.corp$ env JENKINS_HOME=$PWD java -jar jenkins.war \
    --httpPort=1920
...
Information: Jenkins is fully up and running

We can now point a web browser to http://ci.acme.corp:1920/ and install the Git and xUnit plugins (cf. DESCRIPTION section) by following Jenkins' user guide.

Server Jobs

Now that we have Jenkins CI running on http://ci.acme.corp:1920/, let's go back our project, fetch the Jenkins CLI JAR:

my-project$ wget http://ci.acme.corp:1920/jnlpJars/jenkins-cli.jar
...
Saving to: 'jenkins-cli.jar'

check that it can connect to the server:

my-project$ java -jar jenkins-cli.jar -s http://ci.acme.corp:1920/ help
  build
    Builds a job, and optionally waits until its completion.
...

and create the jobs:

my-project$ veda new-jenkins-job --create --auto-trigger     \
  --cli-jar jenkins-cli.jar --url http://ci.acme.corp:1920/  \
  --job-name my-project-%s foo bar
    Server Create Job my-project-foo
    Server Create Job my-project-bar

Pointing a browser to http://ci.acme.corp:1920/, we can see both of them fail within five minutes... because they don't contain any Git commit. Let's remedy that:

my-project$ (cd pkg/foo/ && git add . && git commit -m 'Initial commit')
[master (root-commit) 0c32f10] Initial commit
 2 files changed, 80 insertions(+)
 create mode 100644 foo-test.ils
 create mode 100644 foo.ils
my-project$ (cd pkg/bar/ && git add . && git commit -m 'Initial commit')
[master (root-commit) 33add61] Initial commit
 2 files changed, 81 insertions(+)
 create mode 100644 bar-test.ils
 create mode 100644 bar.ils

Once again, we can see both jobs fail within five minutes. Examining the job's Console Output tabs, we can see that the tests were run but failed:

=== RUN Test_MyFooHelloReturnValue
*** ERROR *** Test failures occured.

=== RUN Test_MyBarHelloReturnValue
*** ERROR *** Test failures occured.

Which is to be expected, as they also fail in our workarea:

my-project$ veda test-package foo
    Testing foo
=== RUN Test_MyFooHelloReturnValue
--- FAIL: Test_MyFooHelloReturnValue
        Single word: Got t instead of expected 42.
          Args: ("You")

Let's fix that:

my-project$ sed -e 's/want 42/want t/' pkg/foo/foo-test.ils > tmp
my-project$ mv tmp pkg/foo/foo-test.ils
my-project$ (cd pkg/foo/ && git commit -m 'Fix test' foo-test.ils)
[master af31f61] Fix test
 1 file changed, 1 insertion(+), 1 deletion(-)

and within five minutes, the job's status panel and Console Output tells us that all is well:

Triggering a new build of my-project-bar #3
Finished: SUCCESS

Note how using --auto-trigger during job creation means that Jenkins CI tries to build bar again after this successful update to foo; this is due to the @requires foo in pkg/bar/bar.ils.

VEDA(1) Crosstwine Veda 0.2.0

Damien Diederen

January 14, 2013