jscpd

JSCPD (JavaScript Copy-Paste Detector) is a sophisticated copy-paste detection tool that scans codebases across multiple programming languages to identify duplicated code blocks. It helps maintain code quality by finding excessive code duplication that can lead to maintenance issues and technical debt.

Key Features:

Multi-Language Support: Detects copy-paste across 170+ programming languages including JavaScript, TypeScript, Python, Java, C#, and more
Flexible Detection Algorithms: Multiple algorithms for finding duplicates including token-based and semantic matching
Comprehensive Reporting: Generates detailed reports in HTML, JSON, XML, Markdown, and other formats with visual diff highlighting
Configurable Thresholds: Set minimum clone sizes, similarity percentages, and detection sensitivity for different file types
Blame Integration: Shows Git blame information to identify when and who introduced duplicated code
Performance Optimization: Efficient scanning algorithms that can handle large codebases with millions of lines
Flexible Ignore Patterns: Sophisticated filtering with glob patterns, file extensions, and content-based exclusions
Statistical Analysis: Provides detailed metrics on duplication levels, affected files, and codebase health indicators

If you need to ignore folders, files or file extensions, use glob expressions ignore property of local .jscpd.json file

Example:

{
  "threshold": 0,
  "reporters": ["html", "markdown"],
  "ignore": [
    "**/node_modules/**",
    "**/.git/**",
    "**/*.md",
    "**/myFolderToSkip/**"
  ]
}

jscpd documentation

Version in MegaLinter: 4.0.5
Visit Official Web Site
See How to configure jscpd rules
- If custom .jscpd.json config file isn't found, .jscpd.json will be used
See How to disable jscpd rules in files

Configuration in MegaLinter

Enable jscpd by adding COPYPASTE_JSCPD in ENABLE_LINTERS variable
Disable jscpd by adding COPYPASTE_JSCPD in DISABLE_LINTERS variable

Variable	Description	Default value
COPYPASTE_JSCPD_ARGUMENTS	User custom arguments to add in linter CLI call Ex: `-s --foo "bar"`
COPYPASTE_JSCPD_COMMAND_REMOVE_ARGUMENTS	User custom arguments to remove from command line before calling the linter Ex: `-s --foo "bar"`
COPYPASTE_JSCPD_CLI_LINT_MODE	Override default CLI lint mode ⚠️ As default value is project, overriding might not work - `file`: Calls the linter for each file - `list_of_files`: Call the linter with the list of files as argument - `project`: Call the linter from the root of the project	`project`
COPYPASTE_JSCPD_PRE_COMMANDS	List of bash commands to run before the linter	None
COPYPASTE_JSCPD_POST_COMMANDS	List of bash commands to run after the linter	None
COPYPASTE_JSCPD_UNSECURED_ENV_VARIABLES	List of env variables explicitly not filtered before calling COPYPASTE_JSCPD and its pre/post commands	None
COPYPASTE_JSCPD_CONFIG_FILE	jscpd configuration file nameUse `LINTER_DEFAULT` to let the linter find it	`.jscpd.json`
COPYPASTE_JSCPD_RULES_PATH	Path where to find linter configuration file	Workspace folder, then MegaLinter default rules
COPYPASTE_JSCPD_DISABLE_ERRORS	Run linter but consider errors as warnings	`false`
COPYPASTE_JSCPD_DISABLE_ERRORS_IF_LESS_THAN	Maximum number of errors allowed	`0`
COPYPASTE_JSCPD_CLI_EXECUTABLE	Override CLI executable	`['jscpd']`

MegaLinter Flavors

This linter is available in the following flavors

Flavor	Description	Embedded linters
all	Default MegaLinter Flavor	126
c_cpp	Optimized for pure C/C++ projects	56
ci_light	Optimized for CI items (Dockerfile, Jenkinsfile, JSON/YAML schemas,XML	22
cupcake	MegaLinter for the most commonly used languages	87
documentation	MegaLinter for documentation projects	49
dotnet	Optimized for C, C++, C# or VB based projects	64
dotnetweb	Optimized for C, C++, C# or VB based projects with JS/TS	73
go	Optimized for GO based projects	51
java	Optimized for JAVA based projects	54
javascript	Optimized for JAVASCRIPT or TYPESCRIPT based projects	59
php	Optimized for PHP based projects	54
python	Optimized for PYTHON based projects	65
ruby	Optimized for RUBY based projects	50
rust	Optimized for RUST based projects	50
salesforce	Optimized for Salesforce based projects	54
swift	Optimized for SWIFT based projects	50
terraform	Optimized for TERRAFORM based projects	54

Behind the scenes

How are identified applicable files

If this linter is active, all files will always be linted

How the linting is performed

jscpd is called once on the whole project directory (project CLI lint mode)

filtering can not be done using MegaLinter configuration variables,it must be done using jscpd configuration or ignore file (if existing)
VALIDATE_ALL_CODEBASE: false doesn't make jscpd analyze only updated files

Example calls

jscpd --output ./report/copy-paste/ .

jscpd --output ./report/copy-paste/ -c .jscpd.json .

Help content

Usage: jscpd [options] <path ...>

detector of copy/paste in files

Options:
  -V, --version              output the version number
  -l, --min-lines [number]   min size of duplication in code lines (Default is
                             5)
  -k, --min-tokens [number]  min size of duplication in code tokens (Default is
                             50)
  -x, --max-lines [number]   max size of source in lines (Default is 1000)
  -z, --max-size [string]    max size of source in bytes, examples: 1kb, 1mb,
                             120kb (Default is 100kb)
  -t, --threshold [number]   threshold for duplication, in case duplications >=
                             threshold jscpd will exit with error
  -c, --config [string]      path to config file (Default is .jscpd.json in
                             <path>)
  -i, --ignore [string]      glob pattern for files what should be excluded
                             from duplication detection
  --ignore-pattern [string]  Ignore code blocks matching the regexp patterns
  -r, --reporters [string]   reporters or list of reporters separated with
                             comma to use (Default is time,console)
  -o, --output [string]      reporters to use (Default is ./report/)
  -m, --mode [string]        mode of quality of search, can be "strict", "mild" and "weak" (Default is "function mild(token) {
    return strict(token) && token.type !== "empty" && token.type !== "new_line";
  }")
  -f, --format [string]      format or formats separated by comma (Example
                             php,javascript,python)
  -p, --pattern [string]     glob pattern to file search (Example **/*.txt)
  -b, --blame                blame authors of duplications (get information
                             about authors from git)
  -s, --silent               do not write detection progress and result to a
                             console
  --store [string]           use for define custom store (e.g. --store leveldb
                             used for big codebase)
  -a, --absolute             use absolute path in reports
  -n, --noSymlinks           dont use symlinks for detection in files
  --ignoreCase               ignore case of symbols in code (experimental)
  -g, --gitignore            ignore all files from .gitignore file
  --formats-exts [string]    list of formats with file extensions
                             (javascript:es,es6;dart:dt)
  -d, --debug                show debug information, not run detection
                             process(options list and selected files)
  -v, --verbose              show full information during detection process
  --list                     show list of total supported formats
  --skipLocal                skip duplicates in local folders, just detect
                             cross folders duplications
  --exitCode [number]        exit code to use when code duplications are
                             detected
  -h, --help                 display help for command

Installation on mega-linter Docker image

Dockerfile commands :

# renovate: datasource=npm depName=jscpd
ARG NPM_JSCPD_VERSION=4.0.5

NPM packages (node.js):
- jscpd@4.0.5