Apigeelint NPM

apigeelint

Node LastCommit CommitActivity Downloads

Static code analysis for Apigee proxy and sharedflow bundles to encourage API developers to use best practices and avoid anti-patterns.

This utility is intended to capture the best practices knowledge from across Apigee including our Global Support Center team, Customer Success, Engineering, and our product team in a tool that will help developers create more scalable, performant, and stable API bundles using the Apigee DSL.

Status

At this point, we are focused on plugin execution and modelling the various lintable assets including Bundles, Proxies, SharedFlows, Targets, Flows, Steps, and Policies.

Plugins that test these abstractions are being developed concurrently.

Reporters (the means to report out results), Ingesters (bundle loaders) are to be developed with Filesystem being the only supported means of loading a bundle and all reporting now going to console.

Installation

You can install apigeellint using npm. But, there is a minimum version of npm required.

First verify the version of npm:
```
npm --version
```
If the version is 8.3.0 or later, then proceed to step 2. If the version is less than 8.3.0, then update:
```
npm install npm@8.3.0 -g
```
Alternatively, you may choose to get the latest npm:
```
npm install npm@latest -g
```
Then install apigeelint:
```
npm install -g apigeelint
```

Usage

Help

apigeelint -h
Usage: apigeelint [options]

Options:
  -V, --version                           output the version number
  -s, --path <path>                       Path of the proxies
  -f, --formatter [value]                 Specify formatters (default: json.js)
  -w, --write [value]                     file path to write results
  -e, --excluded [value]                  The comma separated list of tests to exclude (default: none)
  -x, --externalPluginsDirectory [value]  Relative or full path to an external plugins directory
  -q, --quiet                             do not emit the report to stdout. (can use --write option to write to file)
  --list                                  do not execute, instead list the available plugins and formatters
  --maxWarnings [value]                   Number of warnings to trigger nonzero exit code (default: -1)
  --profile [value]                       Either apigee or apigeex (default: apigee)
  -h, --help                              output usage information

Example:

apigeelint -s sampleProxy/apiproxy -f table.js

Where -s points to the apiProxy source directory and -f is the output formatter desired.

Possible formatters are: "json.js" (the default), "stylish.js", "compact.js", "codeframe.js", "codeclimate.js", "html.js", "table.js", "unix.js", "visualstudio.js", "checkstyle.js", "jslint-xml.js", "junit.js" and "tap.js".

More Examples

Using External Plugins:

apigeelint -x ./externalPlugins -s path/to/your/apiproxy -f table.js

Where -x points to the directory containing externally developed plugins.

You could, for example, create your own plugin for naming conventions, and exclude the builtin plugin that enforces naming conventions (PO007) with the -e option:

apigeelint -x ./externalPlugins -e PO007 -s path/to/your/apiproxy -f table.js

This would effectively override the built-in naming conventions that apigeelint checks.

Excluding plugins

You can, of course, exclude plugins without providing a replacement implementation:

apigeelint -s path/to/your/apiproxy -f table.js -e PO007,ST003

The above would exclude the policy naming convention check (PO007), and would also not check for conditions on an ExtractVariables with a JSONPayload (ST003), if for some reason you wanted to do that.

Writing output to a file

apigeelint -s sampleProxy/apiproxy -f table.js -w existing-outputdir --quiet

The -w option can point to an existing directory, in which case the output will be emitted to a file named apigeelint.out in that directory, in whatever format you specify with -f. An existing file by that name will be overwritten. If the -w option is not a directory, it is treated as the name of a file, and output is written there.

If you do not also specify --quiet the report will go to both stdout and to the specified filesystem destination.

Listing plugins

List plugins and formatters, with or without --externalPluginsDirectory.

apigeelint --list
apigeelint --list -x ./externalPlugins

# or

apigeelint --list --externalPluginsDirectory ./externalPlugins

Selecting a profile

Apigee X/hybrid is very similar to Apigee Edge, but there are differences in the supported policy types, and some of the supported configuration options. For example, policies like GraphQL, the AssertCondition, or the Integration policy step types are available only in X/hybrid.

As a result of these differences, a proxy that is valid in Apigee Edge might not work in Apigee X, and vice versa. Apigeelint uses the --profile option to allow the user to configure which target environment is intended: Edge (--profile apigee) or X/hybrid (--profile apigeex). The default is apigee.

# lint a proxy that will be used in Apigee X/hybrid
apigeelint -f table.js --profile apigeex -s path/to/your/apiproxy

# lint a proxy that will be used in Apigee Edge
apigeelint -f table.js --profile apigee -s path/to/your/apiproxy

As an example, if you lint a proxy that uses the GraphQL policy type, and you specify the apigee profile, the PO028 plugin will issue an error, telling you that the GraphQL policy is not available in the apigee profile. If you lint the same proxy with the apigeex profile, apigeelint will not generate an error.

The selection of a profile affects other checks, too. For example, the Google Authentication feature is available only in X/hybrid.

Pipeline lint job integration

GitLab CI/CD

On GitLab CI/CD, on your .gitlab-ci.yml you can use codequality report artifact to get a report supported by GitLab. Once the CI/CD has been completed, a new tab appears in your Pipeline named "Code Quality". This new tab lets you easily view the information from the apigeelint job, with the associated severity level and lines affected. A widget with the same information appears during merge requests.

apigeelint:
  stage: lint
  image: node:12-alpine
  before_script:
    - npm install -g apigeelint
  script:
    - apigeelint -f codeclimate.js > apigeelint-results.json
  artifacts:
    reports:
      codequality:
        - "${CI_PROJECT_DIR}/apigeelint-results.json"

Does this tool just lint or does it also check style?

This tool does both traditional linting (looking for problematic patterns) and style checking (enforcement of conventions). You can use it for both.

Tests

The test directory includes scripts to exercise a subset of rules. Overall linting can be tested with:

apigeelint -s ./test/fixtures/resources/sampleProxy/24Solver/apiproxy/

This sample exhibits many bad practices and as such generates numerous errors and warnings in output.

In a development installation, the equivalent to the command above is:

node ./cli.js  -s ./test/fixtures/resources/sampleProxy/24Solver/apiproxy/

Contributing

We welcome pull requests for bug fixes and new features. In lieu of a formal style guide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code.

Run the unit tests like this:

npm run test

or, run a subset of tests like this:

./node_modules/mocha/bin/mocha --grep PO033

You can also contribute by reporting issues, asking for new features.

Rules

The list of rules is a work in progress. We expect it to increase over time. As product features change (new policies, deprecated policies, etc), we will change rules as well.

This is the current list:

Linter	Status	Code	Name	Description
Bundle
	:white_check_mark:	BN001	Bundle folder structure correctness.	Bundles have a clear structure. This plugin ignores some files, like .DS_store and any file ending in ~.
	:white_medium_square:	BN002	Extraneous files.	Ensure each folder contains appropriate resources in the bundle.
	:white_check_mark:	BN003	Cache Coherence	A bundle that includes cache reads should include cache writes with the same keys.
	:white_medium_square:	BN004	Unused variables.	Within a bundle variables created should be used in conditions, resource callouts, or policies.
	:white_check_mark:	BN005	Unattached policies.	Unattached policies are dead code and should be removed from production bundles.
	:white_check_mark:	BN006	Bundle size - policies.	Large bundles are a symptom of poor design. A high number of policies is predictive of an oversized bundle.
	:white_check_mark:	BN007	Bundle size - resource callouts.	Large bundles are a symptom of poor design. A high number of resource callouts is indicative of underutilizing out of the box Apigee policies.
	:white_medium_square:	BN008	IgnoreUnresolvedVariables and FaultRules	Use of IgnoreUnresolvedVariables without the use of FaultRules may lead to unexpected errors.
	:white_check_mark:	BN009	Statistics Collector - duplicate policies	Warn on duplicate policies when no conditions are present or conditions are duplicates.
	:white_check_mark:	BN010	Missing policies	Issue an error if a referenced policy is not present in the bundle.
	:white_check_mark:	BN011	Check each XML file for well-formedness.
	:white_check_mark:	BN012	unreferrenced Target Endpoints	Check that each TargetEndpoint can be reached.
	:white_check_mark:	BN013	Unreferenced resources.	Warn for resources that not referenced in any policy. Unreferenced resources are dead code.
Proxy Definition
	:white_check_mark:	PD001	RouteRules to Targets	RouteRules should map to defined Targets.
	:white_check_mark:	PD002	Unreachable Route Rules - defaults	Only one RouteRule should be present without a condition.
	:white_check_mark:	PD003	Unreachable Route Rules	RouteRule without a condition should be last.
	:white_check_mark:	PD004	ProxyEndpoint name	ProxyEndpoint name should match basename of filename.
	:white_check_mark:	PD005	VirtualHost	ProxyEndpoint should have one HTTPProxyConnection, and in the case of profile=apigeex, no VirtualHost.
Target Definition
	:white_check_mark:	TD001	Mgmt Server as Target	Discourage calls to the Management Server from a Proxy via target.
	:white_check_mark:	TD002	Use Target Servers	Encourage the use of target servers.
	:white_check_mark:	TD003	TargetEndpoint name	TargetEndpoint name should match basename of filename.
	:white_check_mark:	TD004	TargetEndpoint SSLInfo	TargetEndpoint HTTPTargetConnection should enable TLS/SSL.
	:white_check_mark:	TD005	TargetEndpoint SSLInfo references	TargetEndpoint SSLInfo should use references for KeyStore and TrustStore.
Flow
	:white_check_mark:	FL001	Unconditional Flows	Only one unconditional flow will get executed. Error if more than one was detected.
Step
	:white_check_mark:	ST001	Empty Step	Empty steps clutter the bundle.
	:white_check_mark:	ST002	Step Structure	each Step should have at most one Name element, one Condition element, no others.
	:white_check_mark:	ST003	Extract Variables Step with JSONPayload	A check for message content should be performed before policy execution.
	:white_check_mark:	ST004	Extract Variables Step with XMLPayload	A check for message content should be performed before policy execution.
	:white_check_mark:	ST005	Extract Variables Step with FormParam	A check for message content should be performed before policy execution.
	:white_check_mark:	ST006	JSON Threat Protection Step	A check for message content should be performed before policy execution.
	:white_check_mark:	ST007	XML Threat Protection Step	A check for message content should be performed before policy execution.
Policy
	:white_check_mark:	PO006	Policy Name & filename agreement	Policy name attribute should coincide with the policy filename.
	:white_check_mark:	PO007	Policy Naming Conventions - type indication	It is recommended that the policy name use a prefix or follow a pattern that indicates the policy type.
	:white_check_mark:	PO008	Policy DisplayName & DisplayName agreement	Check that the policy filename matches the display name of the policy.
	:white_medium_square:	PO009	Service Callout Target - Mgmt Server	Targeting management server may result in higher than expected latency; use with caution.
	:white_medium_square:	PO010	Service Callout Target - Target Server	Encourage use of target servers.
	:white_medium_square:	PO011	Service Callout Target - Dynamic URLs	Error on dynamic URLs in target server URL tag.
	:white_check_mark:	PO012	AssignMessage/AssignTo	Warn on unnecessary AssignTo in AssignMessage when createNew is false and no destination variable.
	:white_check_mark:	PO013	Resource Call Out - Javascript	JSHint, ESLint.
	:white_medium_square:	PO014	Resource Call Out - Java	PMD, Checkstyle.
	:white_medium_square:	PO015	Resource Call Out - Python	Pylint.
	:white_medium_square:	PO016	Statistics Collector - duplicate variables	Warn on duplicate variables.
	:white_medium_square:	PO017	Misconfigured - FaultRules/Fault Rule in Policy	FaultRules are configured in ProxyEndpoints and TargetEndpoints.
	:white_check_mark:	PO018	Regex Lookahead/Lookbehind are Expensive - Threat Protection Policy	Regular expressions that include lookahead or lookbehind perform slowly on large payloads and are typically not required.
	:white_check_mark:	PO019	Reserved words as variables - ServiceCallout Request	Using "request" as the name of a Request may cause unexpected side effects.
	:white_check_mark:	PO020	Reserved words as variables - ServiceCallout Response	Using "response" as the name of a Response may cause unexpected side effects.
	:white_medium_square:	PO021	Statistics Collector - reserved variables	Warn on insertion of duplicate variables.
	:white_check_mark:	PO022	Nondistributed Quota	When using nondistributed quota the number of allowed calls is influenced by the number of Message Processors (MPs) deployed. This may lead to higher than expected transactions for a given quota as MPs now autoscale.
	:white_check_mark:	PO023	Quota Policy Reuse	When the same Quota policy is used more than once you must ensure that the conditions of execution are mutually exclusive or that you intend for a call to count more than once per message processed.
	:white_check_mark:	PO024	Cache Error Responses	By default the ResponseCache policy will cache non 200 responses. Either create a condition or use policy configuration options to exclude non 200 responses.
	:white_check_mark:	PO025	EsLint Errors	Runs EsLint on all policy resources.
	:white_check_mark:	PO026	AssignVariable Usage	With AssignMessage/AssignVariable, check various usage issues. Example: The Name element must be present. The Ref element, if any, should not be surrounded in curlies. And so on.
	:white_check_mark:	PO027	HMAC Usage	With HMAC, check that the SecretKey is present and that a ref= attribute refers to a private variable.
	:white_check_mark:	PO028	Policy Availability in profile	Check for policies available in particular profiles.
	:white_check_mark:	PO029	Known policy type	Check that all policies are of a known type.
	:white_check_mark:	PO030	ExpirySettings	ExpirySettings should use exactly one child element, no deprecated elements.
	:white_check_mark:	PO031	AssignMessage content-type	When assigning to Payload, you should also assign content-type, exactly once.
	:white_check_mark:	PO032	CORS policy hygiene	In a CORS policy, wildcard origins should generate a warning. And other hygiene checks.
	:white_check_mark:	PO033	ExtractVariables policy hygiene	In an ExtractVariables policy, check variable types and other hygiene.
	:white_check_mark:	PO034	AssignMessage policy hygiene	In an AssignMessage policy, check element placement and other hygiene.
FaultRules
	:white_check_mark:	FR001	No Condition on FaultRule	Use Condition elements on FaultRules, unless it is the fallback rule.
	:white_check_mark:	FR002	DefaultFaultRule Structure	DefaultFaultRule should have only supported child elements, at most one AlwaysEnforce element, and at most one Condition element.
	:white_check_mark:	FR003	single FaultRule	When a single FaultRule is present, consider using a DefaultFaultRule.
Conditional
	:white_check_mark:	CC001	Literals in Conditionals	Warn on literals in any conditional statement.
	:white_medium_square:	CC002	Null Blank Checks	Blank checks should also check for null conditions. (to be reviewed)
	:white_check_mark:	CC003	Long condition statement	Conditions should not be long.
	:white_check_mark:	CC004	Overly complex condition	Condition complexity should be limited to fix number of variables and conjunctions.
	:white_check_mark:	CC005	unterminated strings in Condition	Strings within a Condition element must be properly wrapped by double quotes.
	:white_check_mark:	CC006	Detect logical absurdities	Conditions should not have internal logic conflicts - warn when these are detected.
	:white_check_mark:	CC007	Check validity of expression syntax	Condition expressions should use valid syntax. No single quotes, no extraneous or unmatched parens, etc.
Endpoints
	:white_check_mark:	EP001	CORS Policy attachment	Check for multiple CORS policies, or attachment to Target Endpoint.
	:white_check_mark:	EP002	Misplaced Elements	Check for commonly misplaced configuration elements in Proxy and Target Endpoints.
Features
	:white_check_mark:	FE001	Use of Authentication element	Check for the Authentication element in policies or in Target Endpoints.
Deprecation
	:white_check_mark:	DC001	ConcurrentRateLimit Policy Deprecation	Check usage of deprecated policy ConcurrentRateLimit.
	:white_check_mark:	DC002	OAuth V1 Policies Deprecation	Check usage of deprecated OAuth V1 policies.

From an implementation perspective, the focus is on plugin support and flexibility over performance. Compute is cheap.

Release Notes

Release v2.31.0

Condition checks around policies

In release v2.31.0, the plugins PO001, PO002, PO003, PO004, and PO005 have been converted to ST006, ST007, ST003, ST004, and ST005, respectively. These plugins move from the "Policy" category to the "Step" category because the plugin analyzes the attachment of the policy in a Step element, rather than the policy itself. Also these plugins will now generate warnings, rather than errors.

If previously you excluded PO003 via the --excluded option, you must now exclude ST003, and so on.

Sharedflows

Starting with release v2.31.0, using apigeelint against Sharedflows will generate a correct report. Previously the report on a sharedflow was truncated and omitted some warnings and errors.

Support

If you find issues, file a ticket here on Github. Keep in mind that there is no service level agreement (SLA) for responses to these issues. Assume all responses are on an ad-hoc, volunteer basis.

If you simply have questions, we recommend asking on the Apigee forum on GoogleCloudCommunity.com. Apigee experts regularly check that forum.

Apigee customers should use formal support channels for Apigee product related concerns.

License and Copyright

Disclaimer

This tool is open-source software. It is not an officially supported Google product. It is not a part of Apigee, or any other officially supported Google Product.

API bundle lint linter Apigee Edge