TestArchitect Queue command line tool

TestArchitect Queue command line tool, also known as TAQueue, is in charge of generating jobs, which contains a number of TA automated tests in the form of TA batches files, and submitting them to external batch queuing systems, such as JPPF (Java Parallel Processing Framework).

Note: 


TAQueue, dispatches TA batch files which containing test modules or test suites to be executed in the following manners:

  1. TAQueue defines whether a bat file is a test suite or test module
  2. TAQueue parses the test module/test suite from bat files and obtains the following information:
    • repository server and its port number,
    • user account, and user password
  3. TAQueue accesses to repository by information obtained via TAUtilities, then obtain a list of test modules included in the test suite. Note that, jobs are equivalent to batch files, or test modules.
  4. TAQueue generates jobs, and submits jobs to batch queuing systems, such as JPPF (Java Parallel Processing Framework).
  5. The queuing system organizes jobs of a queue. The system then chooses when and which nodes will run the jobs according to system policies.

Commands in TAQueue

create command

. Description

This command is used to generates jobs, and then submit them to external batch queuing systems.

. Synopsis

  • On Windows:
    • run.bat create -<ShortArgumenName> <argumentValue>
    • run.bat create --<LongArgumenName> <argumentValue>
  • On Linux and macOS:
    • run.sh create -<ShortArgumenName> <argumentValue>
    • run.sh create --<LongArgumenName> <argumentValue>

. Arguments

NameDescription
-b (short name)--batch (long name)

Specifies the folder that contains TA batch files.

Usage: 
--batch "<pathToFolder>"
-b "<pathToFolder>”
-s (short name)
--server (long name)		Specifies IP address and port of the batch queuing system server.

Usage:
--server "localhost:11111"
-s "localhost:11111"


-n (short name)
--name (long name)		Assigns a logical name to a submitted queue.
Usage:
--name “Queue01”
-n “Queue01”
 
Notes: It is highly recommended that you assign a unique name to prevent duplicate queue names.
 -f (short name)
--filter (long name)		(Optional) Specifies the filter conditions to determine whether a set of jobs will be executed on which nodes.
The below example is for JPPF’s filter conditions.
Usage:
-f “<ExecutionPolicy><AtLeast><Property>jppf.processing.threads</Property><Value>2</Value></AtLeast></ExecutionPolicy>"

Notes:
- <ExecutionPolicy> tag is optional.
-t (short name)
--timeout (long name)		(Optional) Specifies the maximum wait time for a submitted queue (Units: minutes).
Usage:
--timeout "50"
-t “50”
Notes:
  • Defaults to 60 minutes
  • You can configure this default value at {TAQueue _DIR}\config\ta_dispatcher.properties
-spt (short name)
--split (long name)		(Optional) Chooses "yes/no" option to determine whether TAQueue breaks a batch into multiple test modules for job generation purpose.
Usage:
-- split "yes"
- spt "no"

Notes:
  • Defaults to yes.
  • With this argument specified, the current job file will be broken into a given number of sub-jobs. This number is determined based on the number of included test modules.
  • For example: The current job contains 5 test modules. This argument will split the job into 5 sub-jobs, each contains a single test module.

-jt (short name)
--jobtimeout (long name)

(Optional) Defines a timeout value for each job of a queue
while the job is being executed on the JPPF server.
Unit: minute


Usage:
–jt "30"
--jobtimeout "30"


Note: Defaults to unlimited

-br (short name)
--broadcast (long name)

(Optional) Broadcasts all jobs of a queue to all JPPF’s nodes
Chooses "yes/no" option to determine whether TAQueue
broadcasts a batch to all JPPF’s nodes.
Usage:
–br "yes"
--broadcast "yes"


Note:
· Defaults to no

-pr (short name)
--precondition (long name)

(Optional) Defines a condition before the execution of the
batch file of a job.
Specifies the location folder that contains batch files.
Usage:
–pr "\\data-server\ShareFolder"
--precondition "\\data-server\ShareFolder"

-po (short name)
--postcondition (long name)

(Optional) Defines a condition after the execution of the
batch file of a job.
Specifies the location folder that contains batch files.
Usages:
–po "\\data-server\ShareFolder"
--postcondition "\\data-server\ShareFolder"


Note: When the connection between the TAQueue and the queuing system server is broken, the default timeout for the job execution is 60 minutes. Once the timeout expires, the create command is aborted.


wait command

. Description

Waits whether dispatched jobs are executed entirely.

Notes

  • TAQueue submits jobs to batch queuing systems in asynchronous transfer. Specifically, jobs will be submitted to the queuing systems continuously without waiting for whether the jobs execution is finished or not.

  • When you use the wait command after the create command, TAQueue waits until the timeout, defined in the wait command, expires or the job execution is done, before TAQueue executes the next command.

  • wait is usually used after create.

. Synopsis

  • On Windows:
    • run.bat wait -<ShortArgumenName> <argumentValue>
    • run.bat wait --<LongArgumenName> <argumentValue>
  • On Linux and macOS:
    • run.sh wait -<ShortArgumenName> <argumentValue>
    • run.sh wait --<LongArgumenName> <argumentValue>

. Arguments


NameDescription
-n (short name)
--name (long name)		Name of the submitted queue.

Usage:
--name "Job01"
-n "Job01"
-s (short name)
--server (long name)		Specifies IP address and port of the batch queuing system server.

Usage:
-- server "localhost:11198"
-s "localhost:11198"
Notes:
  • On JPPF, since the JMX (Java Management Extensions) technology is employed for the wait command, the default port number must be 11198.
  • If you’d like to use another port number:
  • Go to the JPPF-x.y.z-driver\config\jppf-driver.properties, folder. (Follow here to see where this file is located.)
  • Comment out the line jppf.management.port by removing the # at the start of the line.
  • Change into a desired port number.
-t (short name)
--timeout (long name)		Specifies the maximum wait time for the queue's status (Units: minutes).
Usage:
--timeout "60"

Notes:
  • Defaults to 60 minutes
  • You can configure this default value at {TAQueue _DIR}\config\ta_dispatcher.properties


. Troubleshooting
As long as you use the wait command after the create command in the broadcast mode (--broadcast= "yes"), the system will return/display the number of jobs incorrectly.

The issue arises from the fact the server utilizes the following formula in the broadcast mode:

        Displayed number of jobs = true number of jobs * (1 + number of nodes)

For detailed information, refer to https://www.jppf.org/tracker/tbg/jppf/issues/JPPF-544


cancel command

. Description

This command is used to cancel all jobs of a specific queue or all queues which are submitted to the JPPF server.


. Synopsis

  • On Windows:
    • run.bat cancel -<ShortArgumenName> <argumentValue>
    • run.bat cancel --<LongArgumenName> <argumentValue>
  • On Linux and macOS:
    • run.sh cancel -<ShortArgumenName> <argumentValue>
    • run.sh cancel --<LongArgumenName> <argumentValue>

. Arguments


NameDescription
-n (short name)
--name (long name)		(Optional) Cancels all jobs of a particular queue on the JPPF
server.
Usage:
–n "Queue1 "
--name "Queue1"
-s (short name)
--server (long name)		Specifies IP address and port of the batch queuing system server.

Usage:
-- server "localhost:11198"
-s "localhost:11198"
-t (short name)
--timeout (long name)		(Optional) Specifies the maximum wait time for a cancelled queue.
Unit: minute
Usage:
--timeout "50
-t "50"
Note:
· Defaults to 60 minutes
· You can configure this default value at <TAQueue
_DIR>\config\ta_dispatcher.properties
-a (short name)
--all (long name)		(Optional) Cancels all current jobs on the JPPF server.
Usage:
-a
--all


Java Parallel Processing Framework

We’ll take JPPF, a typical batch queuing system, as an example to integrate with TAQueue.

Configuring JPPF

  1. Download the following JPPF components.
    • JPPF server/driver distribution
    • JPPF node distribution
    • JPPF administration and monitoring console
    • JPPF application template.
  2. Unzip them into a desired location.
  3. Comment out the line “jppf.discovery.enabled” by removing the # at the start of the line. Note: “ppf.discovery.enabled” is located at the below places, the location where you store downloaded JPPF components.
    • JPPF-x.y.z-node\config\jppf-node.properties
    • JPPF-x.y.z-driver\config\jppf-driver.properties
    • JPPF-x.y.z-application-template\config\jppf.properties
    • JPPF-x.y.z-admin-ui\config\jppf-gui.properties
  4. Change the value of ppf.discovery.enabled of all configuration files, mentioned above, from true to false.

  5. Copy the TAQueueLib.jar file from the <TAQueue_DIR>\plugins folder to the <JPPFx.y.z-driver>/lib folder.

  6. To start a server:

    • Go to “JPPF-x.y.z-driver” folder and open a command prompt or shell console.
    • Run startDriver.bat on Windows or ./startDriver.sh on Linux/macOS.

    • The server is now ready to process job requests.

  7. To start a node:
    • Go to the “JPPF-x.y.z-node” folder and open a command prompt or shell console.
    • Run startNode.bat on Windows or ./startNode.sh on Linux/macOS.
    • Together, this node and the server constitute the JPPF grid.
  8. To start the admin GUI:
    • Go to the “JPPF-x.y.z-admin-ui” folder and open a command prompt or shell console.
    • Type start Console.bat on Windows or ./startConsole.sh on Linux/macOS.

* Important Notes *

In order to avoid the Apache Log4j Vulnerability, please follow these steps:

  1. Download the log4j version 2.17.1 that fixed the issue and extract it
  2. Copy 3 files (log4j-1.2-api-2.17.1.jar ; log4j-api-2.17.1.jar ; log4j-core-2.17.1.jar) to
    1. <JPPFx.y.z-driver>/lib folder
    2. <JPPFx.y.z-node>/lib folder
  3. Delete/Remove the log4j-1.2.15.jarfile at
    1. <JPPFx.y.z-driver>/lib folder
    2. <JPPFx.y.z-node>/lib folder

Running submitted job requests

Ensure that the following requirements are met:

  1. Go to the location where you store TAQueue.
  2. Generate jobs from a TA test suite by using the create command. This job will be submitted to the JPPF server. For example:
    • Location where to store TA batch files: D:\taqueue
    • JPPF server: 192.168.167.60
    • Port number of the server: 11111
    • Name of the submitted job to be assigned: TS1

  3. Upon the successful submission, you should see “Exit code 0 - Success: The queue
    command line executed successfully”.

  4. The server then distributes the jobs to the nodes, that is, the machines which will run the automated tests.

Tip: You can also find the log file at the following location {TAQueue_DIR\log}