jscpd
JSCPD browses all files of your repository to detect abusive copy-pastes
If you need to ignore folders,files or file extensions, use glob expressions ignore property of local .jscpd.json file
Example:
{
"threshold": 0,
"reporters": ["html", "markdown"],
"ignore": [
"**/node_modules/**",
"**/.git/**",
"**/*.md",
"**/myFolderToSkip/**"
]
}
jscpd documentation
- Version in Mega-Linter: 3.3.26
- Visit Official Web Site
- See How to configure jscpd rules
- If custom
.jscpd.jsonconfig file is not found, .jscpd.json will be used
- If custom
- See How to disable jscpd rules in files
Configuration in Mega-Linter
- Enable jscpd by adding
COPYPASTE_JSCPDin ENABLE_LINTERS variable - Disable jscpd by adding
COPYPASTE_JSCPDin DISABLE_LINTERS variable
| Variable | Description | Default value |
|---|---|---|
| COPYPASTE_JSCPD_ARGUMENTS | User custom arguments to add in linter CLI call Ex: -s --foo "bar" |
|
| COPYPASTE_JSCPD_FILE_EXTENSIONS | Allowed file extensions. "*" matches any extension, "" matches empty extension. Empty list excludes all filesEx: [".py", ""] |
Exclude every file |
| COPYPASTE_JSCPD_FILE_NAMES_REGEX | File name regex filters. Regular expression list for filtering files by their base names using regex full match. Empty list includes all files Ex: ["Dockerfile(-.+)?", "Jenkinsfile"] |
Include every file |
| COPYPASTE_JSCPD_PRE_COMMANDS | List of bash commands to run before the linter | None |
| COPYPASTE_JSCPD_POST_COMMANDS | List of bash commands to run after the linter | None |
| COPYPASTE_JSCPD_CONFIG_FILE | jscpd configuration file nameUse LINTER_DEFAULT to let the linter find it |
.jscpd.json |
| COPYPASTE_JSCPD_RULES_PATH | Path where to find linter configuration file | Workspace folder, then Mega-Linter default rules |
| COPYPASTE_JSCPD_DISABLE_ERRORS | Run linter but consider errors as warnings | false |
| COPYPASTE_JSCPD_DISABLE_ERRORS_IF_LESS_THAN | Maximum number of errors allowed | 0 |
Mega-Linter Flavours
This linter is available in the following flavours
| Flavor | Description | Embedded linters | Info | |
|---|---|---|---|---|
 |
all | Default Mega-Linter Flavor | 94 |   |
| dart | Optimized for DART based projects | 41 |   |
|
| documentation | Mega-Linter for documentation projects | 40 |   |
|
| dotnet | Optimized for C, C++, C# or VB based projects | 47 |   |
|
| go | Optimized for GO based projects | 42 |   |
|
| java | Optimized for JAVA based projects | 42 |   |
|
| javascript | Optimized for JAVASCRIPT or TYPESCRIPT based projects | 49 |   |
|
| php | Optimized for PHP based projects | 44 |   |
|
| python | Optimized for PYTHON based projects | 49 |   |
|
| ruby | Optimized for RUBY based projects | 41 |   |
|
| rust | Optimized for RUST based projects | 41 |   |
|
| salesforce | Optimized for Salesforce based projects | 43 |   |
|
| scala | Optimized for SCALA based projects | 41 |   |
|
| swift | Optimized for SWIFT based projects | 41 |   |
|
| terraform | Optimized for TERRAFORM based projects | 45 |   |
Behind the scenes
How are identified applicable files
- If this linter is active, all files will always be linted
How the linting is performed
jscpd is called once on the whole project directory
- filtering can not be done using Mega-Linter configuration variables,it must be done using jscpd configuration or ignore file (if existing)
VALIDATE_ALL_CODEBASE: falsedoes not make jscpd analyze only updated files
Example calls
jscpd --output ./report/copy-paste/ .
jscpd --output ./report/copy-paste/ -c .jscpd.json .
Help content
Usage: jscpd [options] <path ...>
detector of copy/paste in files
Options:
-V, --version output the version number
-l, --min-lines [number] min size of duplication in code lines (Default is
5)
-k, --min-tokens [number] min size of duplication in code tokens (Default is
50)
-x, --max-lines [number] max size of source in lines (Default is 1000)
-z, --max-size [string] max size of source in bytes, examples: 1kb, 1mb,
120kb (Default is 100kb)
-t, --threshold [number] threshold for duplication, in case duplications >=
threshold jscpd will exit with error
-c, --config [string] path to config file (Default is .jscpd.json in
<path>)
-i, --ignore [string] glob pattern for files what should be excluded
from duplication detection
-r, --reporters [string] reporters or list of reporters separated with
comma to use (Default is time,console)
-o, --output [string] reporters to use (Default is ./report/)
-m, --mode [string] mode of quality of search, can be "strict", "mild" and "weak" (Default is "function mild(token) {
return strict(token) && token.type !== 'empty' && token.type !== 'new_line';
}")
-f, --format [string] format or formats separated by comma (Example
php,javascript,python)
-p, --pattern [string] glob pattern to file search (Example **/*.txt)
-b, --blame blame authors of duplications (get information
about authors from git)
-s, --silent do not write detection progress and result to a
console
--store [string] use for define custom store (e.g. --store leveldb
used for big codebase)
-a, --absolute use absolute path in reports
-n, --noSymlinks dont use symlinks for detection in files
--ignoreCase ignore case of symbols in code (experimental)
-g, --gitignore ignore all files from .gitignore file
--formats-exts [string] list of formats with file extensions
(javascript:es,es6;dart:dt)
-d, --debug show debug information, not run detection
process(options list and selected files)
-v, --verbose show full information during detection process
--list show list of total supported formats
--skipLocal skip duplicates in local folders, just detect
cross folders duplications
-h, --help display help for command
Installation on mega-linter Docker image
- NPM packages (node.js):
Example success log
Results of jscpd linter (version 3.3.22)
See documentation on https://nvuillam.github.io/mega-linter/descriptors/copypaste_jscpd/
-----------------------------------------------
[SUCCESS] .automation/test/copypaste/good
┌────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ go │ 2 │ 12 │ 52 │ 0 │ 0 (0%) │ 0 (0%) │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total: │ 2 │ 12 │ 52 │ 0 │ 0 (0%) │ 0 (0%) │
└────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 0 clones.
HTML report saved to report/copy-paste/html/
Detection time:: 42.298ms
Example error log
Results of jscpd linter (version 3.3.22)
See documentation on https://nvuillam.github.io/mega-linter/descriptors/copypaste_jscpd/
-----------------------------------------------
[ERROR] .automation/test/copypaste/bad
Clone found (go):
- copypaste_bad_1.go [1:1 - 33:2] (32 lines, 222 tokens)
copypaste_bad_2.go [1:1 - 33:2]
┌────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ go │ 2 │ 64 │ 444 │ 1 │ 32 (50%) │ 222 (50%) │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total: │ 2 │ 64 │ 444 │ 1 │ 32 (50%) │ 222 (50%) │
└────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 1 clones.
HTML report saved to report/copy-paste/html/
ERROR: jscpd found too many duplicates (50%) over threshold (0%)
Error: ERROR: jscpd found too many duplicates (50%) over threshold (0%)
at ThresholdReporter.report (/node_modules/@jscpd/finder/dist/reporters/threshold.js:12:19)
at /node_modules/@jscpd/finder/dist/in-files-detector.js:82:26
at Array.forEach (<anonymous>)
at /node_modules/@jscpd/finder/dist/in-files-detector.js:81:28