jscpd

JSCPD (JavaScript Copy-Paste Detector) is a sophisticated copy-paste detection tool that scans codebases across multiple programming languages to identify duplicated code blocks. It helps maintain code quality by finding excessive code duplication that can lead to maintenance issues and technical debt.

Key Features:

Multi-Language Support: Detects copy-paste across 170+ programming languages including JavaScript, TypeScript, Python, Java, C#, and more
Flexible Detection Algorithms: Multiple algorithms for finding duplicates including token-based and semantic matching
Comprehensive Reporting: Generates detailed reports in HTML, JSON, XML, Markdown, and other formats with visual diff highlighting
Configurable Thresholds: Set minimum clone sizes, similarity percentages, and detection sensitivity for different file types
Blame Integration: Shows Git blame information to identify when and who introduced duplicated code
Performance Optimization: Efficient scanning algorithms that can handle large codebases with millions of lines
Flexible Ignore Patterns: Sophisticated filtering with glob patterns, file extensions, and content-based exclusions
Statistical Analysis: Provides detailed metrics on duplication levels, affected files, and codebase health indicators

If you need to ignore folders, files or file extensions, use glob expressions ignore property of local .jscpd.json file

Example:

{
  "threshold": 0,
  "reporters": ["html", "markdown"],
  "ignore": [
    "**/node_modules/**",
    "**/.git/**",
    "**/*.md",
    "**/myFolderToSkip/**"
  ]
}

jscpd documentation

Version in MegaLinter: 5.0.11
Visit Official Web Site
See How to configure jscpd rules
- If custom .jscpd.json config file isn't found, .jscpd.json will be used
See How to disable jscpd rules in files

Configuration in MegaLinter

Enable jscpd by adding COPYPASTE_JSCPD in ENABLE_LINTERS variable
Disable jscpd by adding COPYPASTE_JSCPD in DISABLE_LINTERS variable

Variable	Description	Default value
COPYPASTE_JSCPD_ARGUMENTS	User custom arguments to add in linter CLI call Ex: `-s --foo "bar"`
COPYPASTE_JSCPD_COMMAND_REMOVE_ARGUMENTS	User custom arguments to remove from command line before calling the linter Ex: `-s --foo "bar"`
COPYPASTE_JSCPD_CLI_LINT_MODE	Override default CLI lint mode ⚠️ As default value is project, overriding might not work - `file`: Calls the linter for each file - `list_of_files`: Call the linter with the list of files as argument - `project`: Call the linter from the root of the project	`project`
COPYPASTE_JSCPD_PRE_COMMANDS	List of bash commands to run before the linter	None
COPYPASTE_JSCPD_POST_COMMANDS	List of bash commands to run after the linter	None
COPYPASTE_JSCPD_UNSECURED_ENV_VARIABLES	List of env variables explicitly not filtered before calling COPYPASTE_JSCPD and its pre/post commands	None
COPYPASTE_JSCPD_CONFIG_FILE	jscpd configuration file name Use `LINTER_DEFAULT` to let the linter find it	`.jscpd.json`
COPYPASTE_JSCPD_RULES_PATH	Path where to find linter configuration file	Workspace folder, then MegaLinter default rules
COPYPASTE_JSCPD_DISABLE_ERRORS	Run linter but consider errors as warnings	`false`
COPYPASTE_JSCPD_DISABLE_ERRORS_IF_LESS_THAN	Maximum number of errors allowed	`0`
COPYPASTE_JSCPD_CLI_EXECUTABLE	Override CLI executable	`['jscpd']`

MegaLinter Flavors

This linter is available in the following flavors

Flavor	Description	Embedded linters
all	Default MegaLinter Flavor	137
c_cpp	Optimized for pure C/C++ projects	60
ci_light	Optimized for CI items (Dockerfile, Jenkinsfile, JSON/YAML schemas,XML	24
cupcake	MegaLinter for the most commonly used languages	93
documentation	MegaLinter for documentation projects	53
dotnet	Optimized for C, C++, C# or VB based projects	68
dotnetweb	Optimized for C, C++, C# or VB based projects with JS/TS	77
go	Optimized for GO based projects	55
java	Optimized for JAVA based projects	58
javascript	Optimized for JAVASCRIPT or TYPESCRIPT based projects	63
php	Optimized for PHP based projects	58
python	Optimized for PYTHON based projects	70
ruby	Optimized for RUBY based projects	54
rust	Optimized for RUST based projects	54
salesforce	Optimized for Salesforce based projects	57
swift	Optimized for SWIFT based projects	54
terraform	Optimized for TERRAFORM based projects	57

Behind the scenes

How are identified applicable files

If this linter is active, all files will always be linted

How the linting is performed

jscpd is called once on the whole project directory (project CLI lint mode)

filtering can not be done using MegaLinter configuration variables,it must be done using jscpd configuration or ignore file (if existing)
VALIDATE_ALL_CODEBASE: false doesn't make jscpd analyze only updated files

Example calls

jscpd --output ./report/copy-paste/ .

jscpd --output ./report/copy-paste/ -c .jscpd.json .

Help content

Copy/Paste Detector — find duplicated code

Usage: cpd [OPTIONS] [PATH]...

Arguments:
  [PATH]...  Paths to scan for duplicates

Options:
  -k, --min-tokens <MIN_TOKENS>
          Minimum number of tokens to consider a duplicate
  -l, --min-lines <MIN_LINES>
          Minimum number of lines to consider a duplicate
  -x, --max-lines <MAX_LINES>
          Maximum number of lines per block to consider
  -m, --mode <MODE>
          Detection mode: mild, weak, strict
      --skip-comments
          Alias for --mode weak (skip comment tokens)
  -f, --format <FORMAT>
          List of file extensions/formats to check (comma-separated)
  -i, --ignore <IGNORE>
          File-level glob patterns to ignore, e.g. "**/node_modules/**" (comma-separated)
      --ignore-pattern <IGNORE_PATTERN>
          Code-level regex patterns to skip matching tokens during detection, e.g. "//\\s*cpd-disable" (comma-separated)
  -r, --reporters <REPORTERS>
          Output reporters (comma-separated): console,json,xml,csv,html,markdown,badge,sarif,ai,xcode,threshold,silent,console-full Aliases: "full" and "consoleFull" are accepted for "console-full"
  -o, --output <OUTPUT>
          Output directory for file reporters
  -c, --config <CONFIG>
          Path to config file (.jscpd.json)
      --exit-code [<EXIT_CODE>]
          Exit with code if duplicates found (default code: 1)
  -t, --threshold <THRESHOLD>
          Maximum duplication percentage before exit 1
  -b, --blame
          Enrich clones with git blame data
      --no-gitignore
          Do not respect .gitignore files
      --follow-symlinks
          Follow symbolic links
  -z, --max-size <MAX_SIZE>
          Skip files larger than SIZE (e.g. 1kb, 1mb, 100kb, or raw bytes). Default: 1mb
      --workers <WORKERS>
          Number of worker threads (default: auto)
      --no-colors
          Disable ANSI color output
  -a, --absolute
          Use absolute paths in reports
      --ignore-case
          Ignore case of symbols in code (experimental)
      --formats-exts <FORMATS_EXTS>
          Custom format-to-extension mappings (e.g. javascript:es,es6;dart:dt)
      --formats-names <FORMATS_NAMES>
          Custom format-to-filename mappings (e.g. makefile:Makefile,GNUmakefile;docker:Dockerfile)
  -p, --pattern <PATTERN>
          Glob pattern to find files to scan (e.g. **/*.ts, **/*.{js,ts})
      --list
          List all supported formats and exit
      --skip-local
          Skip clones where both fragments are in the same directory [aliases: --skipLocal]
      --min-duplicated-lines <MIN_DUPLICATED_LINES>
          Minimum percentage of duplication to report (0-100) [default: 0]
  -s, --silent
          Do not write detection progress and result to console
      --no-tips
          Do not print tips and promotional messages after detection
      --debug
          Print merged config (CLI + config file) as JSON and exit without running detection
  -h, --help
          Print help
  -V, --version
          Print version

Installation on mega-linter Docker image

Dockerfile commands :

# renovate: datasource=npm depName=jscpd
ARG NPM_JSCPD_VERSION=5.0.11

NPM packages (node.js):
- jscpd@5.0.11