Alexander Brett

Running all tests during a Salesforce.com deployment

21 August 2014

Salesforce.com deployments to a production environment require all unit tests to be run at the time of deployment, and roll back if any of them encounter an error. This is a desirable behaviour for a continuous integration environment, but although the deployment tools provide a runAllTests = true option for sandbox deployments, it has the disadvantage that it runs tests in all namespaces - and depending how many managed packages you have, that could be a very lengthy process indeed.

Following on from the last post though, it turns out that there is a way to achieve exactly this behaviour by gratuitous use of javascript within the ant deploy script. The idea is this: the SFDC deploy task can have nested runTest elements, but natively in ant we can’t dynamically generate them. However, if we can get a list of test classes, we can use javascript to add them to a dynamically generated deploy task.

This approach does have limitations: you can only use it when you are deploying every test class on your org, because this won’t run tests which aren’t in your deployment, but it will calculate code coverage based on which tests are run.

I’ll start where we left off by defining simple build.xml, build.properties, and deploy.js files:

###build.xml

<project name="ant javascript example">
  <property file="build.properties"/>
  
  <target name="deploy">
    <deployscript/>
  </target>
  
  <scriptdef name="deployscript"
    language="javascript"
    src="deploy.js"/>
</project>

###deploy.js

var env  = project.getProperty("e");
username = project.getProperty(env + ".username");
password = project.getProperty(env + ".password");
url      = project.getProperty(env + ".url");

if (!(username && password && url)) {
  fail = project.createTask("fail");
    fail.setMessage("Either you didn't specify an environment, or "
      + "the specified environment didn't have the correct properties "
      + "defined in build.properties.");
  fail.execute();
}

var task = java.lang.Class.forName("com.salesforce.ant.DeployTask").newInstance();
task.setTaskName("SF deploy");
task.setPassword(password);
task.setUsername(username);
task.setServerURL(url);
task.setDeployRoot("myPkg");
task.setProject(project);
task.setMaxPoll(project.getProperty("sf.maxpoll"));

if (project.getProperty("v")) task.setCheckOnly(true);
if (project.getProperty("t")) task.setRunAllTests(true);

task.execute();

###build.properties

dev.username  = user@example.com.dev
dev.password  = passwordToken
dev.serverurl = https://test.salesforce.com

qa.username   = user@example.com.qa
qa.password   = passwordToken
qa.serverurl  = https://test.salesforce.com

prod.username = user@example.com
prod.password = passwordToken
prod.serverurl= https://login.salesforce.com

sf.maxPoll   = 200

Now, we need a way to find all of the test classes. Since I have grep installed on my computer and it’s a great cross-platform solution, I’ll use a quick-and-dirty grep through the files. This assumes you’ve got all of your classes on your hard drive in the src/classes directory, which will be the case if you use the eclipse Force.com IDE or similar, and/or if you use source control. All tests have to be denoted with @isTest or testMethod, so I’m going to do a search on those terms. This would catch any file with testMethod in a comment, for instance, so it’s not perfect, but it’s pretty good.

Once we’ve got that output, we’ll store it in a variable to use in our javascript later on.

###build.xml

<project name="ant javascript example">
  <property file="build.properties"/>
  
  <target name="deploy">
    <exec executable = "grep"
      outputproperty = "testClasses">
      <arg value = "-lEr"/>
      <arg value = "@is[tT]est|test[mM]ethod"/>
      <arg value = "src/classes/"/>
    </exec>
    <deployscript/>
  </target>
  
  <scriptdef name="deployscript"
    language="javascript"
    src="deploy.js"/>
</project>

If we now dive back into the ant-salesforce.jar source code and look in com/salesforce/ant/DeployTast.class, there’s a class called CodeNameElement, and a method called addRunTest. These are what we need to use to run the tests. Just like the DeployTask itself, we can instatiate CodeNameElement with java.lang.Class.forName().newInstance(), and add that instance as a child:

if (project.getProperty("t")) {
  var tests = (project.getProperty(testClasses)|| '').split("\n"));
  
  tests = tests
    .map   (function(e,i,a) {
      return (m = /(\w+)\.cls/.exec(e)) ? m[0] : e; // get just the test names
    }).filter (function(e,i,a) {
      return e  && a.indexOf(e) == i // uniqueness filter 
    });
  
  for each (test in tests) {
    var codeNameElement = java.lang.Class.forName(
        "com.salesforce.ant.DeployTask$CodeNameElement"
      ).newInstance();
    codeNameElement.addText(test);
    task.addRunTest(codeNameElement);
  }
};

This is really ready for use at this point, but for my own satisfaction we can make this even more javascript-oriented. If you crack open lib/ant.jar in your ant installation, you’ll see ExecTask.class halfway down. This is the underlying class for <exec/>, and the only thing different about this class is that in order to append a child ‘’ element, you need to use ‘createArg().setValue()’. Putting this into action, we can finish bringing all the logic into javascript - this javascript works with the build.xml defined at the top of this post.

###deploy.js

function getTestClasses(){
	var propertyName = "testClassesList";
	var exec = project.createTask("exec");
	exec.setExecutable("grep");
	exec.setOutputproperty(propertyName);
	
	exec.createArg().setValue("-lEr");
	exec.createArg().setValue("@is[tT]est|test[mM]ethod");
	exec.createArg().setValue("src/classes/");
	exec.execute();
	
	var result = project.getProperty(propertyName);
	return (typeof result == "string" ? result.split("\n") : [])
		.map   (function(e,i,a){ return (m = /(\w+)\.cls/.exec(e)) ? m[0] : e;})
		.filter(function(e,i,a){ return a.indexOf(e) == i });
}

(function main(){
  var env  = project.getProperty("e");
  username = project.getProperty(env + ".username");
  password = project.getProperty(env + ".password");
  url      = project.getProperty(env + ".url");
  
  if (!(username && password && url)) {
    fail = project.createTask("fail");
    fail.setMessage("Either you didn't specify an environment, or "
      + "the specified environment didn't have the correct properties "
      + "defined in build.properties.");
    fail.execute();
  }
  
  var task = java.lang.Class.forName("com.salesforce.ant.DeployTask")
    .newInstance();
  task.setTaskName("SF deploy");
  task.setPassword(password);
  task.setUsername(username);
  task.setServerURL(url);
  task.setDeployRoot("myPkg");
  task.setProject(project);
  task.setMaxPoll(project.getProperty("sf.maxpoll"));
  
  if (project.getProperty("v")) task.setCheckOnly(true);
  
  if (project.getProperty("t")) {
    for each (test in getTestClasses()) {
      var codeNameElement = java.lang.Class.forName(
          "com.salesforce.ant.DeployTask$CodeNameElement"
        ).newInstance();
      codeNameElement.addText(test);
      task.addRunTest(codeNameElement);
    }
  };
  
  task.execute();
}());

Tags: SFDC

Writing Salesforce ant deployment tasks using Javascript

19 August 2014

The Force.com deployment tool is a .jar file defining some extra tasks such as sf:deploy and sf:retrieve. Examining the example build.xml file we see several calls of the form:

<sf:retrieve 
    username="${sf.username}"
    password="${sf.password}"
    serverurl="${sf.serverurl}"
    maxPoll="${sf.maxPoll}"
    retrieveTarget="retrieveOutput"
    packageNames="MyPkg"/>
    
<sf:bulkRetrieve
    username="${sf.username}"
    password="${sf.password}"
    serverurl="${sf.serverurl}"
    maxPoll="${sf.maxPoll}"
    metadataType="${sf.metadataType}"
    retrieveTarget="retrieveUnpackaged"/>

In a mature development team with several different sandboxes, you may end up wanting to retrieve and deploy from any of several different sandboxes, with or without the bulk API, with or without running all tests1. If you tried to use ant to write logic to decide between them, you’d end up with a great many targets, each with only a slight permutation to the others. What we want to do is make these decisions using Javascript, which will make the code shorter, easier-to-read, and more flexible. We’ll use JDK1.8, and specifically the Nashorn Javascript engine.

Let’s start with the simplest possible build.xml file:

<project name="ant javascript example">
    <property file="build.properties"/>
    <target name="deploy">
        <sf:deploy 
            username="${sf.username}"
            password="${sf.password}"
            serverurl="${sf.serverurl}"
            maxPoll="${sf.maxPoll}"
            deployRoot="mypkg"
            rollbackOnError="true"/>
    </target>
</project>

and a similarly trivial build.properties file:

sf.username  = user@example.com
sf.password  = passwordToken
sf.serverurl = https://login.salesforce.com
sf.maxPoll   = 200

The first thing I want to do is add an optional ‘validate’ flag to my deployment such that when it is set, we only perform a dry-run. The idea is that when you call ant deploy -Dv=1 the deploy task gets the checkonly="true" attribute added. One way to do this natively is:

<project name="ant javascript example">
    <property file="build.properties"/>
    <target name="deploy"
            depends="dryrun,wetrun" >
    </target>
    
    <target name="dryrun"
            if="v">
        <sf:deploy 
            username="${sf.username}"
            password="${sf.password}"
            serverurl="${sf.serverurl}"
            maxPoll="${sf.maxPoll}"
            deployRoot="mypkg"
            checkonly="true"/>
    </target>
    
    <target name="wetrun"
            unless="v">
        <sf:deploy 
            username="${sf.username}"
            password="${sf.password}"
            serverurl="${sf.serverurl}"
            maxPoll="${sf.maxPoll}"
            deployRoot="mypkg"/>
    </target>
</project>

This code is over twice as long, and it’s much harder to follow the flow of what’s happening, and if I subsequently need to add another flag such as -Dt=1 to add runAllTests = "true", I’ll double my code again2. Luckily, ant has our back.

Let’s rip open ant-salesforce.jar and see what’s inside:

  • com
    • salesforce
      • ant
        • BulkRetrieveTask
        • CompileAndTest
        • Configuration
        • ConnectionFactory
        • DeployTask
        • DescribeMetadataTask
        • ListMetadataTask
        • RetrieveTask
        • SFDCAntTask
        • SFDCMDAPIAntTask
        • SFDCMDAPIAntTaskRunner
        • ZipUtil

DeployTask looks quite promising, and we can easily ascertain that DeployTask extends SFDCMDAPIAntTask, which extends SFDCAntTask, which extends Task, and glancing through those classes to find appropriate getters and setters, we can write:

<project name="ant javascript example">
    <property file="build.properties"/>
    
    <target name="deploy">
        <deployscript/>
    </target>
    
    <scriptdef name="deployscript" language="javascript">
      var task = java.lang.Class.forName("com.salesforce.ant.DeployTask")
        .newInstance();
      task.setTaskName("SF deploy");
      task.setPassword(project.getProperty("sf.password"));
      task.setUsername(project.getProperty("sf.username"));
      task.setServerURL(project.getProperty("sf.serverurl"));
      task.setDeployRoot("myPkg");
      task.setProject(project);
      task.setMaxPoll(project.getProperty("sf.maxpoll"));
      
      if (project.getProperty("v")) task.setCheckOnly(true);
      if (project.getProperty("t")) task.setRunAllTests(true);
      
      task.execute();
    </scriptdef>
</project>

Now it’s clear how we can improve and extend this functionality: next, let’s imagine that we might want to deploy to any of arbitrarily many environments (for instance, you’re a sysadmin with dev, QA, and production instances). We want to be able to call ant deploy -De=<env>. Let’s rewrite our build.properties file…

dev.username  = user@example.com.dev
dev.password  = passwordToken
dev.serverurl = https://test.salesforce.com

qa.username   = user@example.com.qa
qa.password   = passwordToken
qa.serverurl  = https://test.salesforce.com

prod.username = user@example.com
prod.password = passwordToken
prod.serverurl= https://login.salesforce.com

sf.maxPoll   = 200

…and adapt our deploy task to use this new format:

<project name="ant javascript example">
    <property file="build.properties"/>
    
    <target name="deploy">
        <deployscript/>
    </target>
    
    <scriptdef name="deployscript" language="javascript">
      <![CDATA[
        var env  = project.getProperty("e");
        username = project.getProperty(env + ".username");
        password = project.getProperty(env + ".password");
        url      = project.getProperty(env + ".url");
        
        if (!(username && password && url)) {
            fail = project.createTask("fail");
            fail.setMessage("Either you didn't specify an environment, or the specified "
              + "environment didn't have the correct properties defined in "
              + "build.properties.local.");
            fail.execute();
        }
        
        var task = java.lang.Class.forName("com.salesforce.ant.DeployTask")
          .newInstance();
        task.setTaskName("SF deploy");
        task.setPassword(password);
        task.setUsername(username);
        task.setServerURL(url);
        task.setDeployRoot("myPkg");
        task.setProject(project);
        task.setMaxPoll(project.getProperty("sf.maxpoll"));
        
        if (project.getProperty("v")) task.setCheckOnly(true);
        if (project.getProperty("t")) task.setRunAllTests(true);
        
        task.execute();
      ]]>
    </scriptdef>
</project>

There are lots of other options to be explored here, but I think this should provide a solid starting point for making powerful deployment scripts that help get the job done.

  1. By default the run all tests flag is quite broken when you have managed packages, in that it runs tests from all namespaces. I’ll write up a solution to that issue in a later post. 

  2. There is a more efficient and complicated way to do this by setting extra properties using <condition/>, but javascript still beats it. 

Tags: SFDC