behat-logger-cli v1.6.0
BehatLoggerExtension
this package provides an extension for behat to log the test results in an json-file. this package also provide commands to validate and merge this json files.
Installation
INFORMATION: if you want to use the behat logger extension for your project, please read the "PHP Integration" section! The installation section installs only the cli-commands as standalone application
npm installation
npm install behat-logger-cli -g
behat-logger-cli --help
manual installation
- download the latest behat-logger-cli.phar from the github releases page
- make the phar-file executable
chmod u+x behat-logger-cli.phar
php behat-logger-cli.phar --help
# optional:
mv behat-logger-cli.phar behat-logger-cli
mv behat-logger-cli /usr/local/bin/
behat-logger-cli --help
docker image
docker run -v /path/to/logs/:/behat/ seretos/behat-logger-cli --help
PHP Integration
add the package to your project as below
composer require seretos/behat-logger-extension --dev
vendor/bin/behat-logger-cli --help
activate the logger in your behat.yml:
default:
formatters:
logger: ~
extensions:
seretos\BehatLoggerExtension\BehatLoggerExtension:
output_path: '%paths.base%/build/behat'
OPTIONAL: if you use an symfony application, you can add this extension (seretos\BehatLoggerExtension\BehatLoggerExtensionBundle) to your Symfony Kernel and integrate the commands in your cli
Command line usage
combine different result jsons into one file and one suite:
behat-logger-cli combine:logs [suite-name] --output=/output/path/ --regex=results/firefox*
if different jsons contain a testresult for one test with the same environment, this command throws an exception
convert all found feature-files to one json-file without results:
behat-logger-cli feature:to:log [suite-name] --output=/output/path/ --regex=features/
check that all scenarios in the log file has an unique title:
This command is deprecated please use validate:scenario:id in future!
behat-logger-cli validate:scenario:title [log-file.json]
check that all schenarios in the log file has an unique id tag:
behat-logger-cli validate:scenario:id [log-file.json] --identifier_tag_regex="/^testrail-case-([0-9]*)$/"
check that all tests are executed in the given environments:
# check that all browserless tests are executed
behat-logger-cli validate:execution actual.json expected.json --tags=~javascript --environments=unknown
# check that all browser tests are executed in firefox and chrome
behat-logger-cli validate:execution actual.json expected.json --tags=javascript --environments=firefox --environments=chrome
send a json-result to testrail and create sections and cases
behat-logger-cli testrail:push:cases testRailSuiteName actual.json
send a json-result to testrail and create environment configurations:
behat-logger-cli testrail:push:configs testRailSuiteName actual.json
send a json-result to testrail and create results
behat-logger-cli testrail:push:results testRailSuiteName actual.json testResultName --milestone=v2.8.0
the commands testrail:push:cases and testrail:push:results requires an .testrail.yml in the current work directory with the following informations:
api:
server: https://yourTestrail.testrail.io/
user: yourUser@mail.de
password: yourPassword
project: youtProject
template: Test Case (Steps)
type: Automated
identifier: custom_preconds
run_group_field: custom_automation_type
# set field an priorities on specific tags
fields:
/^.*$/:
custom_automation_type: Behat
priorities:
/^priority_low$/: Low
log format
first of all, the json file contains the behat suite. if the log-writer can not detect the suite name, they use a suite named "default"
{
"suites": [
{
"name": "default",
"features": {
...
}
}
]
}
the suite contains a list of features:
"features": {
"features\/featurefile.feature": {
"title": "feature title",
"filename": "features\/featurefile.feature",
"description": null,
"language": "en",
"scenarios": {
...
}
},
...
}
and a feature contains scenarios with steps and results
"scenarios": {
"scenariotitle": {
"title": "scenariotitle",
"tags": ["behattag1","behattag2"],
"steps": [
{
"line": 0,
"text": "the user 'test' exists",
"keyword": "Given",
"arguments": []
},
{
"line": 1,
"text": "i logged in as 'test'",
"keyword": "And",
"arguments": []
}
],
"results": [
...
]
},
...
}
and last but not least, contains the features results. the environment property is the browser name. on guette the environment name is setted to "unknown"
"results": [
"firefox": {
"environment": "firefox",
"duration": "1.00",
"message": null,
"stepResults": [
{
"line": 0,
"passed": true,
"screenshot": null,
"message": null
},
{
"line": 1,
"passed": true,
"screenshot": null,
"message": null
}
]
}
]
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago