Test configuration
Configuration to the Pumpo#5 framework is done mainly thanks to configuration files. The main configuration file must be
named exactly config.conf
and must be placed exactly on the following path:
src/test/resources/config.conf
Nested configuration
The configuration file can contain:
- Key value pairs using the syntax
key=value
on each line. In case the value is a string, it must be enclosed in double quotes. - Relative path to another configuration file to be included using the syntax
include "directory/other-config-file.conf"
.
Nesting of configuration files is unlimited. All key value pairs are flattened. In case a key is present several times, the value of the last occurrence will be taken.
This pattern can be used in most complex environments where each person is responsible to maintain configuration for a set of systems under test. By splitting the configuration map into many files it is much easier to manage the configuration using any git flow because merge conflicts will be very limited.
Connection to the testing farm
In most cases the most important configuration is the URL of the testing farm. It needs to be configured under the key
webdriver.url
. For example when using a paid service Browserstack, the configuration would look like:
webdriver.url="https://<user>>:<token>@hub-cloud.browserstack.com/wd/hub"
In case you are running your own farm based on our docker-compose on your computer, then it would look like:
webdriver.url="http://127.0.0.1:4444/wd/hub"
Connection to systems under test
A critical topic is how to manage all configurable values required to connecting to systems under test. Those include mainly:
- URLs
- Authentication credentials
- Timeouts and other non-functional settings The Pumpo#5 library includes a pattern how to manage those settings in configuration files that may or not be included in the versioning system and how to pass the values to the intermediary clients connecting to systems under tests (mainly browsers but also other clients called drivers).
Configuration as session capabilities
One way to configure credentials and other settings when using any browser, emulator or proxy (hereafter called driver) is to pass them as session capabilities using the @Capability annotation as follows:
public class TestClass {
@Test
public void doSomething(@Capability(key = "browserName", value = "driverId")
@Capability(key = "key1", value = "configKey1", type = ValueType.STRING_PROPERTY)
AbcApplication abcApplication) {
...
abcApplication.step1()
.step2();
...
}
}
Where:
- AbcApplication is the selected class of type Application that represents the system under test
- key1 is the key of the setting
- configKey1 is the key under which the value of the setting can be found anywhere in the nested configuration files
- type is the type of the value, ValueType.STRING_PROPERTY in most cases
Usually those key value pairs include some url and optionally authentication details or non-functional settings such as timeouts or connection modes which is specific to each underlying technical stack.
Unlimited number of capabilities can be specified. Capabilities are serialised as a map and passed to the target driver during session initialisation. In case you need to change capabilities during one test, you need to instantiate another application and pass different capabilities to it.
Conditional capabilities
It is possible to specify a capability that will be used only when certain condition (another capability is present, another capability has certain value) is met. For an example check out Conditional capabilities section of Mobile DSL reference.
Configuration as method arguments
Another way to configure connection to systems under test is to pass values as arguments in the methods that are specific for each client (domain). The main example is how URLs are defined for the WEB domain or how endpoints are defined for the REST/SOAP/HTTP domain, see below.
General settings
Pumpo#5 supports following settings (via nested config files):
Key | Type | Description |
---|---|---|
pn5.webdriver.tls.ignore | boolean | Disabled TLS certificates check on farm connections for all applications |
Following capabilities can be applied to any application:
Capability | Type | Description |
---|---|---|
pn5:tls:ignore | boolean | Setting it to 'true' will allow you to ignore issues with TLS certificates on your test farm that is running over https. |
Session settings
Session settings can be passed to the test farm using the @Capability
annotation of whatever proxy host (browser or
any other driver).
Capability | Type | Description |
---|---|---|
sessionTimeout | String | Timeout until the session is killed if there is no activity. It is specified in seconds, e.g. "120s". Default value is "70s". |
name | String | Name of the session that can be used e.g. to specify the name of the tester or of the test itself |
enableVNC | boolean | Enable VNC on the test farm UI - makes sense only for browsers |
pn5:manualSession | boolean | Whether a session should be handled manually |
pn5:keepSession | boolean | Whether the session should be kept open event if test successful or aborted |
pn5:lookup:default | String | Which type should be used as default for element lookup |
pn5:remoteTimeout | Integer | Timeout, in seconds, to wait for a response from an external service. Default value is "70". |
pn5.capabilities.dump | String | Log current capabilities to console |
We do not reproduce documentation of standard capabilities supported by Selenoid, only those that are key based on our experience. Please see an exhaustive list in the reference documentation
pn5:manualSession
When set to true
session opening will not be handled automatically and one needs to call the method sessionStart().
Later on the methods sessionClose() and again sessionStart() can be called to close the session and reopen it later.
This enables to start the session just in time or to close it immediately after using the driver even if the test still
continues. This enables long-running tests to not consume resources on the test farm while not necessary. Note: sessions
are automatically closed if the test is successful or aborted. To keep the session open see the other session related
annotation driver8:keepSession below.
Note: Even if this annotation is an extension of the protocol it will also work with commercial test farms because session is closed from the client side.
pn5:keepSession
When set to true
the session will not be closed even if the test is successful or aborted. By default, any session is
immediately closed after the test is successful or aborted. This helps to keep resource consumption as low as possible.
In case of test failure the configured (or default) session timeout will be used by the test farm even if the client (
test) has disconnected.
Note: Even if this annotation is an extension of the protocol it will also work with commercial test farms because session is closed from the client side.
pn5:lookup:default
What lookup type should be used as default. This value should be remembered and used for all annotations where "by" is set to DEFAULT.
Valid values are "id", "xpath", "name", "text", "className", "description", "automatorCommand", "method" and "default" ( which is application dependent).
Please, bear in mind that not all the lookup types are valid and available, depending on which application type you are testing.
pn5:remoteTimeout
Timeout, in seconds, to wait idle for a response from an HTTP request to an external endpoint or service. After this timeout, the channel will close, and you will see a TimeoutException in the test results.
Default value is 70 (e.g. 70 seconds). It has to be less than or equal to sessionTimeout value (which by default is also 70 seconds).
pn5.capabilities.dump
When set to true
, all the capabilities used by the current session are logged to the console. Default value is true
.
WEB
Selenoid routing
Websites are tested via browsers that run remotely on a test farm. To use such remote browser in a test method the test method needs to be passed as argument a WebApplication annotated with following capabilities:
Key | Type | Mandatory | Description |
---|---|---|---|
browserName | N/A | Mandatory | Value needs to match the test farm configuration. |
Configuration of web urls and credentials for websites / web applications is defined in the page objects representing the application. A page object looks typically as:
@Navigate("https://www.google.com")
public interface SearchPage extends WebDriverAccessor {
@SetValue(value = "q", by = Lookup.NAME)
SearchPage typeIntoSearchBox(String term);
@ExtendedAction
default SearchPage submit() {
getDriver().findElement(By.name("q")).submit();
return null;
}
SearchResultsPage onSearchResultsPage();
}
The annotation @Navigate("https://www.google.com")
contains the configuration of where to find the SearchPage.
Browser logs
Key | Type | Description |
---|---|---|
pn5.webdriver.browser.logs | boolean | In case of exception thrown from test, save browser logs to files (works only with Chrome) |
In case this option is false, you can still access the logs using the webDriver native way using .manage().logs().
HTTP
To use a remote HTTP driver in a test method the test method needs to be passed as argument a HttpApplication annotated with following capabilities:
Key | Type | Mandatory | Description |
---|---|---|---|
browserName | N/A | Mandatory | Value needs to match the test farm configuration. |
browserName
This capability allows routing commands to the right driver. The value needs to be inline with what is configured on
your test farm infrastructure. Our default is to use pn5-driver8
for all areas covered by our universal component
Driver8.
Endpoint
Configuration of endpoints to accessed is passed using one of the domain specific methods prepareRestRequest
,
prepareSoapRequest
or prepareHttpRequest
.
Examples
The following test will connect to the REST endpoint https://foobar.com/api/dogs using the HTTP GET method and assert that the response code is 200.
public class Test {
@Test
void listDogs(@Capability(key = "browserName", value = "pn5-driver8")
HttpApplication restClient) {
restClient
.prepareRestRequest("https://foobar.com/api/dogs", "GET")
.sendAndGetResponse()
.assertStatus(200);
}
}
SQL
To use a remote SQL driver in a test method the test method needs to be passed as argument a SqlApplication annotated with following capabilities:
Key | Type | Mandatory | Description |
---|---|---|---|
browserName | N/A | Mandatory | Value needs to match the test farm configuration. |
pn5:queryTimeout | Integer | Optional | Timeout in seconds waiting for a query to return. |
pn5:sqlDialect | String | Mandatory | The dialect of the database server |
pn5:sqlHostname | String | Mandatory | The hostname of the database server |
pn5:sqlPort | Integer | Mandatory | The port of the database server |
pn5:sqlDatabase | String | Mandatory | The name of the database / schema |
pn5:sqlUsername | String | Optional | Username to authenticate with |
pn5:sqlPassword | String | Optional | Password to authenticate with |
pn5:hidden:sqlPassword | String | Optional | Same as above, but the value will be hidden and protected in all logs. |
browserName
This capability allows routing commands to the right driver. The value needs to be inline with what is configured on
your test farm infrastructure. Our default value is pn5-driver8
for all areas covered by our universal component
Driver8.
pn5:queryTimeout
This capability allows setting a timeout, in seconds, to wait for a query or update statement being executed to return a
result. The default value is 60
seconds.
pn5:sqlDialect
The dialect of the database server to connect to. Possible values are sqlServer
, postgresql
, mysql
and mariadb
pn5:sqlHostname
The hostname of the database server to connect to. Can be a fully qualified domain name or IP address.
pn5:sqlPort
The port on which the database server is listening. Should be an integer.
pn5:sqlUsername
The username to use when authenticating with the database server.
pn5:sqlPassword / pn5:hidden:sqlPassword
The password to use when authenticating with the database server.
Examples
The following test will connect to a Microsoft SQL Server on hostname and port to be found in configuration files under keys sql.host and sql.port. The schema is configured under key sql.db. Dialect is configured under key * *sql.dialect. Authentication will use the user and password to be found in configuration files under keys sql.user and sql.pass**. Query timeout is set to 5 seconds.
public class Test {
@Test
void listRows(@Capability(key = "browserName", value = "pn5-driver8")
@Capability(key = "pn5:sqlDialect", value = "sql.dialect", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sqlHostname", value = "sql.host", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sqlPort", value = "sql.port", type = ValueType.INTEGER_PROPERTY)
@Capability(key = "pn5:sqlDatabase", value = "sql.db", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sqlUsername", value = "sql.user", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sqlPassword", value = "sql.pass", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:queryTimeout", value = "5", type = ValueType.INTEGER)
SqlApplication sqlClient) {
client
.prepareQuery("select count(*) FROM [schemaName].[tableName] WHERE [name] = '#{name}'")
.withParam("name", "John Doe")
.execute();
}
}
MongoDB
To use a remote MongoDB driver, you need to pass an object that implements MongoApplication interface, annotated with following capabilities:
Key | Type | Mandatory | Description |
---|---|---|---|
browserName | N/A | Mandatory | Value needs to match the test farm configuration. |
pn5:mongoHostname | String | Mandatory | The hostname of the database server |
pn5:mongoPort | Integer | Mandatory | The port of the database server |
pn5:mongoDatabase | String | Mandatory | The name of the database to connect to |
pn5:mongoUsername | String | Mandatory | Username to authenticate with |
pn5:mongoPassword | String | Mandatory | Password to authenticate with |
browserName
This capability allows routing commands to the right driver. The value needs to be inline with what is configured on
your test farm infrastructure. Our default value is pn5-driver8
for all areas covered by our universal component
Driver8.
pn5:mongoHostname
The hostname of the database server to connect to. Can be a fully qualified domain name or IP address.
pn5:mongoPort
The port on which the database server is listening. Should be an integer.
pn5:mongoUsername
The username to use when authenticating with the database server.
pn5:mongoPassword
The password to use when authenticating with the database server.
Examples
The following test will connect to a MongoDB on hostname and port to be found in configuration files under keys * mongo.host and mongo.port, and database configured under key mongo.database. Authentication will use the user and password to be found in configuration files under keys mongo.user** and *mongo.password**.
It will then prepare a command to create a new collection, execute and assert the command was successful. Finally, it will prepare another command to list all the user collections on the database, execute, and assert the collection was really created and it exists.
class TestClass {
@Test
void createAndDropCollection(
@Capability(key = BROWSER_NAME, value = BROWSER_NAME_MONGO)
@Capability(key = CAPABILITY_MONGO_HOSTNAME, value = "mongo.host", type = ValueType.STRING_PROPERTY)
@Capability(key = CAPABILITY_MONGO_PORT, value = "mongo.port", type = ValueType.INTEGER_PROPERTY)
@Capability(key = CAPABILITY_MONGO_DATABASE, value = "mongo.database", type = ValueType.STRING_PROPERTY)
@Capability(key = CAPABILITY_MONGO_USERNAME, value = "mongo.user", type = ValueType.STRING_PROPERTY)
@Capability(key = CAPABILITY_MONGO_PASSWORD, value = "mongo.password", type = ValueType.STRING_PROPERTY)
MongoApplication mongoApplication) {
mongoApplication
.prepareCommand()
.createCollection(NEW_COLLECTION)
.execute()
.assertIsSuccessful()
.andThen()
.prepareCommand()
.listCollections()
.execute()
.assertCollectionIsPresent(NEW_COLLECTION);
}
}
SFTP
To use a remote SFTP driver in a test method the test method needs to be passed as argument a SftpApplication annotated with following capabilities:
Key | Type | Mandatory | Description |
---|---|---|---|
browserName | N/A | Mandatory | Value needs to match the test farm configuration. |
pn5:sftpHost | String | Mandatory | The SFTP server host to connect to. |
pn5:sftpPort | String | Mandatory | The SFTP server port to connect to. |
pn5:sftpUsername | String | Mandatory | The username to authenticate with against the SFTP server. |
pn5:sftpPassword | String | Mandatory | The password to authenticate with against the SFTP server. |
pn5:hidden:sftpPassword | String | Mandatory | Same as above, but the value will be hidden and protected in all logs. |
browserName
This capability allows routing commands to the right driver. The value needs to be inline with what is configured on
your test farm infrastructure. Our default is to use pn5-sftp
for all areas covered by our universal component
Driver8.
pn5:sftpHost
The host of the SFTP server. Can be a fully qualified domain name or IP address. No protocol qualifier should be specified.
pn5:sftpPort
The port on which the SFTP server is listening. Beware that communication on the standard port 22 is usually not allowed by default in enterprise environments.
pn5:sftpUsername
The user to authenticate with against the SFTP server.
pn5:sftpPassword / pn5:hidden:sftpPassword
The password to authenticate with against the SFTP server.
Examples
The following test will connect to a remote SFTP server running on the host and port found in the configuration file, under keys sftp.host and sftp.port. The authentication is by username and password, with values found under keys sftp.user and sftp.pass.
It will upload a CSV file found in local path path/to/file/forUpload.csv to remote folder in server /remoteFolder .
public class TestClass {
@Test
public void uploadCsvFileTest(
@Capability(key = "browserName", value = "pn5-sftp")
@Capability(key = "pn5:sftpHost", value = "sftp.host", type = STRING_PROPERTY)
@Capability(key = "pn5:sftpPort", value = "sftp.port", type = INTEGER_PROPERTY)
@Capability(key = "pn5:sftpUser", value = "sftp.user", type = STRING_PROPERTY)
@Capability(key = "pn5:sftpPassword", value = "sftp.pass", type = STRING_PROPERTY)
SftpApplication sftp) {
sftpClient.upload("path/to/file/forUpload.csv", "/remoteFolder");
}
}
KAFKA
To use a remote Kafka driver in a test method the test method needs to be passed as argument a KafkaApplication annotated with following capabilities:
Key | Type | Mandatory | Description |
---|---|---|---|
browserName | N/A | Mandatory | Value needs to match the test farm configuration. |
pn5:kafkaUrl | String | Mandatory | The URL in the form host:port , without any protocol qualifier. |
pn5:kafkaUsername | String | Optional | The username to use when authentication is enabled. In case Kafka accepts anonymous access, the parameter must not be passed. |
pn5:kafkaPassword | String | Optional | The password to use when username has been specified. In case username was not specified, password will be ignored. |
pn5:hidden:kafkaPassword | String | Optional | Same as above, but the value will be hidden and protected in all logs. |
pn5:kafkaGroupId | String | Optional | The consumer group id to be used in case a read query is processed. |
pn5:kafkaProtocol | String | Optional | The value of this determines the security protocol used for communication between clients and brokers. |
browserName
This capability allows routing commands to the right driver. The value needs to be inline with what is configured on
your test farm infrastructure. Our default is to use pn5-kafka
for all areas covered by our universal component
Driver8.
pn5:kafkaUrl
Kafka url can be specified using a hostname, or an IP address and a port in the form host:port
. In case kafka uses
authentication over SSL the certificate provided by Kafka during the handshake must
match the host specified in the configuration.
In case you are using a certificate authority that is not trusted by standard JDK distributions then the driver needs to be enhanced with the certification authority of your choice.
pn5:kafkaUsername + pn5:kafkaPassword / #### pn5:kafkaUsername + pn5:hidden:kafkaPassword
Kafka username and password are used both in authentication using SASL_PLAINTEXT or SASL_SSL with PLAIN mechanism.
pn5:kafkaProtocol
Kafka protocol determines the security protocol used for communication between clients and brokers. The value of this configuration option defines the type of encryption and authentication mechanisms employed in the Kafka cluster. The values you can currently specify are SASL_SSL and SASL_PLAINTEXT. If you don't specify any, SASL_SSL will be chosen by default.
pn5:kafkaGroupId
The consumer group id to be used in case a read query is processed. If the parameter is omitted then manual assignment will be used and this is probably the preferred way to work with Kafka in tests.
Examples
The following test will connect to Kafka on the url that needs to be found in configuration files under the key * *kafka.url** using no authentication (anonymous) access.
public class TestClass {
@Test
public void topicListTest(@Capability(key = "browserName", value = "pn5-driver8")
@Capability(key = "pn5:kafkaUrl", value = "kafka.url", type = ValueType.STRING_PROPERTY)
KafkaApplication kafka) {
kafka
.listTopics()
.assertThatContainsTopicNamed("shouldExist");
}
}
The following test will connect to Kafka on the url that needs to be found in configuration files under the key * *kafka.url using SASL_SSL with PLAIN mechanism authentication with username and password to be found in config files under respective keys kafka.username and kafka.password**. In case a read query is to be processed the groupId "testGroup" will be used.
public class TestClass {
@Test
public void topicListTest(@Capability(key = "browserName", value = "pn5-driver8")
@Capability(key = "pn5:kafkaUrl", value = "kafka.url", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:kafkaUsername", value = "kafka.username", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:kafkaPassword", value = "kafka.password", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:kafkaGroupId", value = "testGroup", type = ValueType.STRING_PROPERTY)
KafkaApplication kafka) {
kafka
.listTopics()
.assertThatDoesNotContainTopicNamed("shouldNotExist");
}
}
Samba
To use a remote Samba driver in a test, the test method needs a SambaApplication parameter annotated with following capabilities:
Key | Type | Mandatory | Description |
---|---|---|---|
browserName | N/A | Mandatory | Value needs to match the test farm configuration. |
pn5:sambaHost | String | Mandatory | The Samba server host to connect to. |
pn5:sambaPort | String | Mandatory | The Samba server port to connect to. |
pn5:sambaUsername | String | Mandatory | The username to authenticate with against the Samba server. |
pn5:sambaPassword | String | Mandatory | The password to authenticate with against the Samba server. |
pn5:hidden:sambaPassword | String | Mandatory | Same as above, but the value will be hidden and protected in all logs. |
pn5:sambaDomain | String | Mandatory | The domain to connect in the Samba server |
pn5:sambaSharename | String | Mandatory | The name of the share to connect in the Samba server |
browserName
This capability allows routing commands to the right driver. The value needs to be inline with what is configured on
your test farm infrastructure. Our default is to use pn5-samba
for all areas covered by our universal component
Driver8.
pn5:sambaHost
The host of the Samba server. Can be a fully qualified domain name or IP address. No protocol qualifier should be specified.
pn5:sambaPort
The port the Samba server is listening to. Beware that communication on the standard port 445 is usually not allowed by default in enterprise environments.
pn5:sambaUsername
The user to authenticate with against the Samba server.
pn5:sambaPassword / pn5:hidden:sambaPassword
The password to authenticate with against the Samba server.
pn5:sambaDomain
The domain to connect to in the Samba server.
pn5:sambaSharename
The name of share where you want to connect to in the Samba server.
Examples
The following test will connect to a Samba server on hostname and port to be found in configuration files under keys * *samba.host and samba.port, use the domain configured under key samba.domain, and the share name configured under key samba.shareName. Authentication will use the user and password to be found in configuration files under keys samba.username and samba.password**.
It will then list the files in the root directory and assert that it contains the default paths . (for current directory) and .. (for previous directory) in the listed folders and files returned.
Finally, it will print information for all the folders and files found in the directory.
public class TestClass {
@Test
public void listFilesInCurrentEmptyDirectory(@Capability(key = "browserName", value = "pn5-samba")
@Capability(key = "pn5:sambaHost", value = "samba.host", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sambaPort", value = "samba.port", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sambaUsername", value = "samba.username", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sambaPassword", value = "samba.password", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sambaDomain", value = "samba.domain", type = ValueType.STRING_PROPERTY)
@Capability(key = "pn5:sambaSharename", value = "samba.shareName", type = ValueType.STRING_PROPERTY)
SambaApplication samba) {
samba
.listFiles("")
.assertThatContainsFileOrFolderNamed(".")
.assertThatContainsFileOrFolderNamed("..")
.printFilesInformation();
}
}