jscpd

JSCPD browses all files of your repository to detect abusive copy-pastes

If you need to ignore folders,files or file extensions, use glob expressions ignore property of local .jscpd.json file

Example:

{
  "threshold": 0,
  "reporters": ["html", "markdown"],
  "ignore": [
    "**/node_modules/**",
    "**/.git/**",
    "**/*.md",
    "**/myFolderToSkip/**"
  ]
}

jscpd documentation

• Version in Mega-Linter: 3.3.26
• Visit Official Web Site
• See How to configure jscpd rules
  • If custom .jscpd.json config file is not found, .jscpd.json will be used
• See How to disable jscpd rules in files

Configuration in Mega-Linter

• Enable jscpd by adding COPYPASTE_JSCPD in ENABLE_LINTERS variable
• Disable jscpd by adding COPYPASTE_JSCPD in DISABLE_LINTERS variable

Variable	Description	Default value
COPYPASTE_JSCPD_ARGUMENTS	User custom arguments to add in linter CLI call Ex: -s --foo "bar"
COPYPASTE_JSCPD_FILE_EXTENSIONS	Allowed file extensions. "*" matches any extension, "" matches empty extension. Empty list excludes all files Ex: [".py", ""]	Exclude every file
COPYPASTE_JSCPD_FILE_NAMES_REGEX	File name regex filters. Regular expression list for filtering files by their base names using regex full match. Empty list includes all files Ex: ["Dockerfile(-.+)?", "Jenkinsfile"]	Include every file
COPYPASTE_JSCPD_PRE_COMMANDS	List of bash commands to run before the linter	None
COPYPASTE_JSCPD_POST_COMMANDS	List of bash commands to run after the linter	None
COPYPASTE_JSCPD_CONFIG_FILE	jscpd configuration file name Use LINTER_DEFAULT to let the linter find it	.jscpd.json
COPYPASTE_JSCPD_RULES_PATH	Path where to find linter configuration file	Workspace folder, then Mega-Linter default rules
COPYPASTE_JSCPD_DISABLE_ERRORS	Run linter but consider errors as warnings	false
COPYPASTE_JSCPD_DISABLE_ERRORS_IF_LESS_THAN	Maximum number of errors allowed	0

IDE Integration

Use jscpd in your favorite IDE to catch errors before Mega-Linter !

	IDE	Extension Name	Install
	Visual Studio Code	vscode-jscpd

Mega-Linter Flavours

This linter is available in the following flavours

	Flavor	Description	Embedded linters	Info
	all	Default Mega-Linter Flavor	93
	dart	Optimized for DART based projects	40
	documentation	Mega-Linter for documentation projects	39
	dotnet	Optimized for C, C++, C# or VB based projects	46
	go	Optimized for GO based projects	41
	java	Optimized for JAVA based projects	41
	javascript	Optimized for JAVASCRIPT or TYPESCRIPT based projects	48
	php	Optimized for PHP based projects	43
	python	Optimized for PYTHON based projects	48
	ruby	Optimized for RUBY based projects	40
	rust	Optimized for RUST based projects	40
	salesforce	Optimized for Salesforce based projects	42
	scala	Optimized for SCALA based projects	40
	swift	Optimized for SWIFT based projects	40
	terraform	Optimized for TERRAFORM based projects	44

Behind the scenes

How are identified applicable files

• If this linter is active, all files will always be linted

How the linting is performed

jscpd is called once on the whole project directory

• filtering can not be done using Mega-Linter configuration variables,it must be done using jscpd configuration or ignore file (if existing)
• VALIDATE_ALL_CODEBASE: false does not make jscpd analyze only updated files

Example calls

jscpd --output ./report/copy-paste/ .

jscpd --output ./report/copy-paste/ -c .jscpd.json .

Help content

Usage: jscpd [options] <path ...>

detector of copy/paste in files

Options:
  -V, --version              output the version number
  -l, --min-lines [number]   min size of duplication in code lines (Default is
                             5)
  -k, --min-tokens [number]  min size of duplication in code tokens (Default is
                             50)
  -x, --max-lines [number]   max size of source in lines (Default is 1000)
  -z, --max-size [string]    max size of source in bytes, examples: 1kb, 1mb,
                             120kb (Default is 100kb)
  -t, --threshold [number]   threshold for duplication, in case duplications >=
                             threshold jscpd will exit with error
  -c, --config [string]      path to config file (Default is .jscpd.json in
                             <path>)
  -i, --ignore [string]      glob pattern for files what should be excluded
                             from duplication detection
  -r, --reporters [string]   reporters or list of reporters separated with
                             comma to use (Default is time,console)
  -o, --output [string]      reporters to use (Default is ./report/)
  -m, --mode [string]        mode of quality of search, can be "strict", "mild" and "weak" (Default is "function mild(token) {
      return strict(token) && token.type !== 'empty' && token.type !== 'new_line';
  }")
  -f, --format [string]      format or formats separated by comma (Example
                             php,javascript,python)
  -p, --pattern [string]     glob pattern to file search (Example **/*.txt)
  -b, --blame                blame authors of duplications (get information
                             about authors from git)
  -s, --silent               do not write detection progress and result to a
                             console
  --store [string]           use for define custom store (e.g. --store leveldb
                             used for big codebase)
  -a, --absolute             use absolute path in reports
  -n, --noSymlinks           dont use symlinks for detection in files
  --ignoreCase               ignore case of symbols in code (experimental)
  -g, --gitignore            ignore all files from .gitignore file
  --formats-exts [string]    list of formats with file extensions
                             (javascript:es,es6;dart:dt)
  -d, --debug                show debug information, not run detection
                             process(options list and selected files)
  -v, --verbose              show full information during detection process
  --list                     show list of total supported formats
  --skipLocal                skip duplicates in local folders, just detect
                             cross folders duplications
  -h, --help                 display help for command

Installation on mega-linter Docker image

• NPM packages (node.js):
  • jscpd

Example success log

Results of jscpd linter (version 3.3.22)
See documentation on https://nvuillam.github.io/mega-linter/descriptors/copypaste_jscpd/
-----------------------------------------------

[SUCCESS] .automation/test/copypaste/good
    ┌────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
    │ Format │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
    ├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
    │ go     │ 2              │ 12          │ 52           │ 0            │ 0 (0%)           │ 0 (0%)            │
    ├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
    │ Total: │ 2              │ 12          │ 52           │ 0            │ 0 (0%)           │ 0 (0%)            │
    └────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
    Found 0 clones.
    HTML report saved to report/copy-paste/html/
    Detection time:: 42.298ms

Example error log

Results of jscpd linter (version 3.3.22)
See documentation on https://nvuillam.github.io/mega-linter/descriptors/copypaste_jscpd/
-----------------------------------------------

[ERROR] .automation/test/copypaste/bad
    Clone found (go):
     - copypaste_bad_1.go [1:1 - 33:2] (32 lines, 222 tokens)
       copypaste_bad_2.go [1:1 - 33:2]

    ┌────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
    │ Format │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
    ├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
    │ go     │ 2              │ 64          │ 444          │ 1            │ 32 (50%)         │ 222 (50%)         │
    ├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
    │ Total: │ 2              │ 64          │ 444          │ 1            │ 32 (50%)         │ 222 (50%)         │
    └────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
    Found 1 clones.
    HTML report saved to report/copy-paste/html/
    ERROR: jscpd found too many duplicates (50%) over threshold (0%)
    Error: ERROR: jscpd found too many duplicates (50%) over threshold (0%)
        at ThresholdReporter.report (/node_modules/@jscpd/finder/dist/reporters/threshold.js:12:19)
        at /node_modules/@jscpd/finder/dist/in-files-detector.js:82:26
        at Array.forEach (<anonymous>)
        at /node_modules/@jscpd/finder/dist/in-files-detector.js:81:28