url-xi v3.3.0
URL-XI - Extended URL Rest Tester
Can be used to test & monitor REST based API-s and ordinary HTML based http(s) requests. Supports all http requests GET,PUT,DELETE,POST, HEAD and OPTIONS.
Url-xi is a simplified version POSTMAN and using a similar concept, but has better support for a flow of request and correlation of request. It can additionally return other metrics than response times of the http requests as the main return value of the test case.
Url-xi has model based testing framework inspired by Mocha and Jest. A test case is built up in several steps. A step contains one or several API requests. Reporting is done on each step and the all all underlying requests. The tool is built for API monitoring from the bottom up. Detailed timings are reported per request. You can customize what should be reported as the result of a request.
System requirements
- Node js : Must be installed
Installation
Clone from Github or install from npm with 'npm install url-xi -g'
Concept
A test case configuration is defined in a JSON file. The configuration can contain several http requests and you can chain them to a flow of request with correlation between the requests. Status and response time is reported for each request. Samples are found in the installation directory.
Extractors
Extractors are used for correlation of request and for validation of response. The following type of extractors are supported:
- JSONPath
- XPath
- Regexp
- JavaScript
"extractors": [
{
"type": "xpath",
"expression": "//*[local-name() = 'GameId']/text()",
"variable": "gameId"
},
{
"type": "regexp",
"expression": "b:GameId>(\\d+)<",
"variable": "gameId2"
},
{
"type": "header",
"expression": "content-type",
"variable": "homePageContent"
},
{
"type": "regexp",
"expression": "<script\\s+src=\"(.+)\">\\s+<\/script>",
"variable": "javaScripts",
"array": true
},
]Special boolean flags that can be used in extractors
- array : Save the array of extracted values. Default is to save a random value in the array.
- index : Save a random index of extracted array instead of the random value.
- counter : Save the size of the extracted array.
Assertions
Are used for validation of response of requests. Assertions works togethers with extractors and variables. Result of assertions are stored in the test result.
"assertions": [
{
"type": "javaScript",
"value": "{{origin}}",
"description": "Returned origin should contain an IP address. Method={{method}}",
"expression": "/^((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\\.(?!$)|$)){4}$/.test(value)",
"failStep": true,
"reportFailOnly":true
}
]Transforms
Transforms are used when you need to transform values of extractors before the can be used for correlation. Transformers are based on regular expressions.
"transformers": [
{
"type":"replace",
"source": "{{videoHostPath}}/{{videoTemplate}}",
"target": "videoChunk",
"from": "$Number$",
"to": "{{$lapIdx1}}"
},
{
"type":"extract",
"source": "{{manifest}}",
"target": "videoHostPath",
"from": "^((?:\/\/|[^\/]+)*(.*))\/"
}
]- Replace: Replace source and place result in variable defined in target
- Extract: Extract from source and place result in variables defined in target. Regular expression with groups in from.
Variables
Variables are used for storing values of extractors or as input parameters.
"variables": [
{
"key": "eventId",
"type": "number",
"usage": "",
"value": "'let arr=[1,2];arr[Math.floor(Math.random() * arr.length)]'"
},
{
"key": "requestIdleTime",
"type": "number",
"usage": "input",
"value": 1000,
"validation": "value > 999 && value <= 15000"
},
{
"key": "capacity",
"type": "number",
"usage": "returnValue",
"value": 0,
"unit": "seats"
},
{
"key": "venueName",
"type": "string",
"usage": "inResponse",
"value": ""
}
]- The usage property is important for variables.
- input: is input variables which can be changed with -i flag in the cli interface. You should include validation of them
- returnValue: Will be the return value of the test run
- inResponse: Is additional information in stored in the test result.
- You can set an initial value with JavaScript. It requires double dots to work. See above
- Validation is also done with JavaScript, but does not require double dots.
Dynamic variables
Variables which are not defined int variables sections are called dynamic variables. They are created by extractors and have keys not defined in the variables section.
Variable placeholders
All variables can be defined with the mustache syntax. A placeholder looks like this {{variableName}}
System variables
- $timestamp - Current timestamp (ms) epoch format
- $name - Name of current test
- $name - Name of current step
- $lap - The lap number when an iterator is used. Starts with 0
- $lapIdx1 - The lap number when an iterator is used. Starts with 1
Random data generation
Random data generation with the NPM module faker is supported. Can be used as variables or be dynamically generated with mustache syntax for placeholders.
{
"key": "email",
"type": "string",
"usage": "inResponse",
"value": "{{$faker.internet.email}}"
}See: https://www.npmjs.com/package/faker for all placeholders
Posting data with x-www-form-urlencoded content type
You can use standard JSON for posting an application/x-www-form-urlencoded form with comma separated values.
{
"name": "Azure Portal Login (OAUTH2)",
"disabled":false,
"requests": [
{
"config": {
"method": "post",
"url": "https://login.microsoftonline.com/{{tentantId}}/oauth2/v2.0/token",
"data": {
"username": "{{username}}",
"password": "{{password}}",
"grant_type": "password",
"client_secret": "{{client_secret}}",
"client_id": "{{client_id}}",
"scope": "https://graph.microsoft.com/.default offline_access"
},
"headers": {
"Content-type": "application/x-www-form-urlencoded",
"Accept": "application/json; charset=utf-8"
}
},
"extractors": [
{
"type": "jsonpath",
"expression": "$.access_token",
"variable": "accessToken"
},
{
"type": "jsonpath",
"expression": "$.refresh_token",
"variable": "refreshToken"
}
],
"assertions": [
{
"type": "javaScript",
"value": "{{accessToken}}",
"description": "Login must return a valid access token.",
"expression": "value !== undefined",
"failStep": true,
"reportFailOnly": true
}
]
}
]
} Here we also show the feature that steps and requests can be disabled with the disabled boolean property
JavaScript support
You can run JavaScript as post and pre processing of request. Javascript can be injected both on step and request level. Often used as another way of extract values from response. Javascript also support the chai style of assertions. Look at the chai assertions api.
Javascript runs in a safe sandbox based on the Node VM with VM2 interface (see npm vm2). You can define the Javascript directly in the json as an string or an array. Javascript can also be defined in external files, when you run url-xi in project mode.
"scripts": [
{
"scope": "before",
"name":"Set vars",
"script": [
"uxs.setVar('var1', new Date())",
"uxs.expect(2).to.equal(2)",
"uxs.assert('foo' !== 'bar', 'foo is not bar')"
]
},
{
"name": "Get User Id",
"options": {
"builtin": [
"os"
]
},
"scope": "after",
"script": "getUserId.js"
}
]Iterator
You can use an iterator to iterate a step in several laps. An iterator can be based on a number or an array. Variable substitution is supported for the value of the iterator. The variable must be of type array and extraction must have the array flag.
Extractor for iterator
{
"type": "regexp",
"expression": "<script\\s+src=\"(.+)\">\\s+<\/script>",
"variable": "javaScripts",
"array": true
}Step Example 1 - Get extracted Java Scripts
{
"name": "Get JavaScripts",
"iterator": {
"varName": "javaScript",
"value": "{{javaScripts}}"
},
"requests": [
{
"config": {
"method": "get",
"url": "{{javaScript}}"
},
"extractors": [],
"notSaveData": true
}
]
},Step example 2 - Run all http methods
{
"name": "HTTP Methods",
"iterator": {
"varName": "method",
"value": [
"get",
"post",
"patch",
"put",
"delete"
]
},
"requests": [
{
"config": {
"method": "{{method}}",
"url": "/{{method}}",
"data": "{\"testdata\":true,\"timestamp\":{{$timestamp}}}"
},
"extractors": [
{
"type": "header",
"expression": "server",
"variable": "server"
},
{
"type": "jsonpath",
"expression": "$.origin",
"variable": "origin"
}
],
"assertions": [
{
"type": "javaScript",
"value": "{{origin}}",
"description": "Returned origin should contain an IP address. Method={{method}}",
"expression": "/^((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\\.(?!$)|$)){4}$/.test(value)",
"failStep": true,
"reportFailOnly":true
}
]
}
]
}Step example 3 - Iterator which polls for valid data
{
"name": "Poll for new result",
"idleBetweenRequests": "{{requestIdleTime}}",
"iterator": {
"value": "{{pollCount}}",
"waitForValidResponse": true
}
}- Iterator count is in value property
- WaitForValidResponse will do polling of requests in step until an assertion with failStep set is returning success
- You also see a new feature here it is idleBetweenRequests. It can be used on test and step level
Request data as a string
Data in request as JSON is supported by default. If data is string format you must use an array for complex requests. Here comes a SOAP example with xml data.
{
"name": "Get Remaining Tickets for Game",
"requests": [
{
"config": {
"method": "post",
"url": "/CheckGamesService.svc",
"data": [
"<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:tem=\"http://tempuri.org/\">",
"<soap:Header xmlns:wsa=\"http://www.w3.org/2005/08/addressing\">",
"<wsa:To>http://sesthbwb09p.apica.local:8001/CheckGamesService.svc</wsa:To>",
"<wsa:Action>http://tempuri.org/ICheckGamesService/RemainingTicketsPerGameId</wsa:Action></soap:Header>",
"<soap:Body>",
"<tem:RemainingTicketsPerGameId>",
"<tem:gameID>{{gameId}}</tem:gameID>",
"<tem:isCachingOff>true</tem:isCachingOff>",
"</tem:RemainingTicketsPerGameId>",
"</soap:Body>",
"</soap:Envelope>"
],
"headers": {
"SOAPAction": "http://tempuri.org/ICheckGamesService/RemainingTicketsPerGameId"
}
},
"extractors": [
{
"type": "xpath",
"expression": "//*[local-name() = 'RemainingTicketsPerGameIdResult']/text()",
"variable": "remainingTickets"
}
]
}
]
}Status code validation for request response.
- Default is that a response status between 200 and 299 is interpreted as a successful request
- You can override that with the expectedStatus array defined in the request section.
{
"expectedStatus": [401],
"config": {
"method": "get",
"url": "/basic-auth/foo/error",
"auth": {
"username": "foo",
"password": "bar"
}
}
}Supported syntax for requests
Url-xi is based on the Axios framework. It means that the axios syntax for request configuration can be used. See: https://www.npmjs.com/package/axios
The test case schema
A test case is validated with by follow JSON schema. It is currently not published on internet. This is for a MacOs/Linux installation. Different directory on Windows. We will soon publish the schema on internet.
"$schema":"file:///usr/local/lib/node_modules/url-xi/config/url-xi-schema-v1-0.json",
"name": "HTTP-Bin Test HTTP Methods",Running an URL-XI test CLI
url-xi -f samples/tm_order_tickets.json
url-xi -h
Usage: url-xi [options]
Options:
-V, --version output the version number
-f, --file <file> test config file
-r, --results <dir> results dir
-xh, --xheaders <headers> extra headers (default: "{}")
-i, --inputs <inputs> input variables. Comma separated list of value pairs var=value format (default: "")
-u, --url <url> base url
-l, --log_level <log_level> log level (default: "info")
-nd, --nodata no response data in report
-po, --parse_only parse json only. No not run
-rn, --result_name <result_name> name of the result
-s, --server start as server
-p, --port <port> server port (default: "8070")
-tc, --time_calc <time_calc> Custom request-time calculation (default: "totalTime")
-h, --help display help for commandSyntax for inputs
url-xi -f samples/default_test.json -i "defIdleTime=1000, requestIdleTime=1000"CLI Console Report
----- Process results [Ticketmonster Home Page] -----
----- [Test Summary] -----
Test Name: Ticketmonster Home Page
Total Response Time: 310
Start Time: 2020-12-18T10:09:00.894Z
End Time: 2020-12-18T10:09:01.222Z
Number of steps: 1
Total Content length: 493
Return value: 310
Result success: true
----- [Steps result] -----
Home page
[success=true, duration=310, content-length=493, start time=2020-12-18T10:09:00.896Z, ignore duration=false]
[GET] http://ticketmonster.apicasystem.com/ticket-monster/
[Success=true, Duration=310.12, Content-length=493,Start time=2020-12-18T10:09:00.896Z, Status=(200 : OK)]
Timings [ Wait=152.18, DNS=6.71, SSL/TLS handshake =0.00 TCP=73.43, FirstByte=83.08, Download=1.42]
Customize Request result reporting
Most API monitoring tools can report total response time of requests and summarize them to a total. Url-xi has a much more advanced feature for this which is tailored for monitoring. It will help you find performance and availability issues from several perspectives. You may want to investigate DNS response time or only the server response time (Time To First Byte). This is the options:
"requestTimeCalculation": {
"description": "Custom calculation of request response time. Default total response time in ms.",
"type": "string",
"enum": [
"TotalTime",
"Request",
"DNS",
"TimeToFirstBuffer",
"DownloadTime",
"ContentLength"
]Running url-xi as a http server
url-xi -s
[2020-08-05T22:30:41.244] [INFO] url-xi - url-xi(1.8.1) started with [
'/usr/local/bin/node',
'/Users/janostgren/work/node/url-xi/dist/cli/index.js',
'-s'
]
[2020-08-05T22:30:41.252] [INFO] url-xi - URL XI server (version 1.1.5) started on http port 8066API
POST http://localhost:8066/api/url-xi/run - Run a test case POST http://localhost:8066/api/url-xi/parse - Parse a test case
Supported query parameter
- nodata = Produce result without data
- baseUrl = change the base URL. Qual as -u parameter in cli interface
- inputs = List of input variables . Example : inputs="api_key=DEMO_KEY"
curl -i -X POST -d @./samples/default_test.json http:/localhost:8066/api/url-xi/parse -H "Content-Type: application/json; charset=UTF-8"
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 24
ETag: W/"18-Uv4N+TqqnzgG/uMPTz4ruEfRYrY"
Date: Wed, 05 Aug 2020 20:40:24 GMT
Connection: keep-alive
{"message":"Parsing ok"}Samples
Samples are found in the installation directory of url-xi. It is the global node directory followed by url-xi/samples
- Linux/Unix : /usr/local/lib/node_modules/url-xi/samples
- Windows : Dynamic. PC global installs happen under %APPDATA%:
$ ls -l /usr/local/lib/node_modules/url-xi/samples
total 80
-rw-r--r-- 1 janostgren admin 4051 26 Okt 1985 app-insight-demo.json
-rw-r--r-- 1 janostgren admin 4579 26 Okt 1985 cldemo_soap_game_service.json
-rw-r--r-- 1 janostgren admin 3015 26 Okt 1985 default_test.json
-rw-r--r-- 1 janostgren admin 10588 26 Okt 1985 http-bin-test.json
-rw-r--r-- 1 janostgren admin 417 26 Okt 1985 tm_home.json
-rw-r--r-- 1 janostgren admin 5944 26 Okt 1985 tm_order_tickets.jsonSimple Example
{
"name": "Ticketmonster Home Page",
"description": "Simple Scenario for the TM home page",
"baseURL": "http://ticketmonster.apicasystem.com",
"steps": [
{
"name": "Home page",
"requests": [
{
"config": {
"url": "/ticket-monster"
}
}
]
}
]
}Advanced Sample - Application Insight
Extract metrics from Microsoft Application Insight with the REST API. Return value of extracted metrics as the result of the test. Look at the variable named pageViewDuration. The example also have advanced json path correlation. You get the index to the variable pageViewIndex and use it the next extractions.
{
"name": "Application Insight Demo",
"description": "Get metrics from Microsoft Application insight",
"variables": [
{
"key": "api_key",
"type": "string",
"usage": "input",
"value": "DEMO_KEY",
"hideValue": true
},
{
"key": "application",
"type": "string",
"usage": "input",
"value": "DEMO_APP"
},
{
"key": "aggregation",
"type": "string",
"usage": "",
"value": "'let arr=['max','min','avg'];arr[Math.floor(Math.random() * arr.length)]'"
},
{
"key": "pageViewName",
"type": "string",
"usage": "inResponse"
},
{
"key": "pageViewIndex",
"type": "number",
"usage": "",
"value":0
},
{
"key": "pageViewDuration",
"type": "number",
"usage": "returnValue",
"value":0
}
],
"baseURL": "https://api.applicationinsights.io",
"config": {
"headers": {
"x-api-key": "{{api_key}}"
}
},
"steps": [
{
"name": "Get Page views",
"requests": [
{
"config": {
"method": "post",
"url": "/v1/apps/{{application}}/metrics",
"data": [
{
"id": "Page Views duration per path for Edge",
"parameters": {
"metricId": "pageViews/duration",
"aggregation": "{{aggregation}}",
"timespan": "PT240H",
"segment": "pageView/name,pageView/urlPath",
"filter": "startswith(client/browser,'Edg')"
}
}
]
},
"extractors": [
{
"type": "jsonpath",
"expression": "$[0].body.value.segments[*].pageView/name",
"variable": "pageViewIndex",
"index":true
},
{
"type": "jsonpath",
"expression": "$[0].body.value.segments[{{pageViewIndex}}].pageView/name",
"variable": "pageViewName"
},
{
"type": "jsonpath",
"expression": "$[0].body.value.segments[{{pageViewIndex}}].segments[0].pageViews/duration.{{aggregation}}",
"variable": "pageViewDuration"
},
{
"type": "jsonpath",
"expression": "$[0].id",
"variable": "id"
}
],
"assertions": [
{
"type": "javaScript",
"value": "{{pageViewDuration}}",
"description": "Duration should be greater than 1 ms",
"expression": "value > 1",
"failStep": true
},
{
"type": "regexp",
"value": "{{aggregation}}",
"description": "Aggregation must be max,min or avg",
"expression": "(max|min|avg)",
"failStep": true
}
]
}
]
}
]
}
Result report example
This example is without result data. The nodata switch is used.
{
"name": "Ticketmonster Order Tickets",
"baseURL": "http://ticketmonster.apicasystem.com",
"flowControl": "Chained Flow",
"returnValue": 1858,
"unit": "ms",
"success": true,
"duration": 1858,
"startTime": 1611130669786,
"endTime": 1611130671668,
"contentLength": 75133,
"timings": {
"wait": 172.9523180000001,
"dns": 7.399875999999722,
"secureHandshake": 0,
"tcp": 478.9874999999997,
"firstByte": 1020.9950390000004,
"download": 185.5268970000002,
"total": 1858.4617540000004
},
"variables": [
{
"key": "email",
"type": "string",
"usage": "inResponse",
"value": "Breana.Murazik29@hotmail.com"
},
{
"key": "bookingId",
"type": "number",
"usage": "inResponse",
"value": 6772882
}
],
"steps": [
{
"name": "Home page",
"success": true,
"duration": 332,
"startTime": 1611130669788,
"endTime": 1611130670129,
"contentLength": 493,
"timings": {
"wait": 172.07248100000015,
"dns": 1.2887349999998605,
"secureHandshake": 0,
"tcp": 69.8254179999999,
"firstByte": 88.60071300000004,
"download": 1.472927000000027,
"total": 331.9715390000001
},
"ignoreDuration": false,
"requests": [
{
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticketmonster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-2.7.1",
"ApicaStep": "Home page"
},
"url": "http://ticketmonster.apicasystem.com/ticket-monster/",
"method": "get",
"success": true,
"duration": 331.9715390000001,
"startTime": 1611130669788,
"endTime": 1611130670129,
"contentLength": 493,
"timings": {
"wait": 172.07248100000015,
"dns": 1.2887349999998605,
"secureHandshake": 0,
"tcp": 69.8254179999999,
"firstByte": 88.60071300000004,
"download": 1.472927000000027,
"total": 331.9715390000001
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Wed, 20 Jan 2021 08:17:50 GMT",
"content-type": "text/html",
"content-length": "493",
"connection": "close",
"last-modified": "Tue, 16 Jul 2019 09:23:34 GMT",
"x-powered-by": "Undertow/1"
},
"error": {}
}
]
},
{
"name": "Get Events",
"success": true,
"duration": 165,
"startTime": 1611130670129,
"endTime": 1611130670298,
"contentLength": 589,
"timings": {
"wait": 0.1398340000000644,
"dns": 1.3458989999999176,
"secureHandshake": 0,
"tcp": 61.9997699999999,
"firstByte": 103.0122960000001,
"download": 0.2584199999998873,
"total": 165.41031999999996
},
"ignoreDuration": false,
"request": [
{
"requestHeaders": {
"Accept": "application/json, text/plain, */*",
"ApicaScenario": "Ticketmonster Order Tickets",
"ApicaCheckId": "92e02d76-63cf-4a54-a4d6-29b9488fdc1a ",
"AppDynamicsSnapshotEnabled": "true",
"User-Agent": "url-xi-2.7.1",
"ApicaStep": "Get Events"
},
"url": "http://ticketmonster.apicasystem.com/ticket-monster/rest/events?_=1611130670129",
"method": "get",
"success": true,
"duration": 165.41031999999996,
"startTime": 1611130670129,
"endTime": 1611130670298,
"contentLength": 589,
"timings": {
"wait": 0.1398340000000644,
"dns": 1.3458989999999176,
"secureHandshake": 0,
"tcp": 61.9997699999999,
"firstByte": 103.0122960000001,
"download": 0.2584199999998873,
"total": 165.41031999999996
},
"status": 200,
"statusText": "OK",
"headers": {
"server": "nginx/1.10.3 (Ubuntu)",
"date": "Wed, 20 Jan 2021 08:17:50 GMT",
"content-type": "application/json",
"content-length": "589",
"connection": "close",
"x-powered-by": "Undertow/1"
},
"error": {}
}
]
},
]
}3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago