Skip to content

jscpd

GitHub stars sarif GitHub release (latest SemVer) GitHub last commit GitHub commit activity GitHub contributors

JSCPD (JavaScript Copy-Paste Detector) is a sophisticated copy-paste detection tool that scans codebases across multiple programming languages to identify duplicated code blocks. It helps maintain code quality by finding excessive code duplication that can lead to maintenance issues and technical debt.

Key Features:

  • Multi-Language Support: Detects copy-paste across 170+ programming languages including JavaScript, TypeScript, Python, Java, C#, and more
  • Flexible Detection Algorithms: Multiple algorithms for finding duplicates including token-based and semantic matching
  • Comprehensive Reporting: Generates detailed reports in HTML, JSON, XML, Markdown, SARIF and other formats with visual diff highlighting. SARIF support is partial: every clone is reported at warning level regardless of its size, so a 3-line duplicate and a 300-line one look identical to a SARIF consumer (upstream issue)
  • Configurable Thresholds: Set minimum clone sizes, similarity percentages, and detection sensitivity for different file types
  • Blame Integration: Shows Git blame information to identify when and who introduced duplicated code
  • Performance Optimization: Efficient scanning algorithms that can handle large codebases with millions of lines
  • Flexible Ignore Patterns: Sophisticated filtering with glob patterns, file extensions, and content-based exclusions
  • Statistical Analysis: Provides detailed metrics on duplication levels, affected files, and codebase health indicators

If you need to ignore folders, files or file extensions, use glob expressions ignore property of local .jscpd.json file

Example:

{
  "threshold": 0,
  "reporters": ["html", "markdown"],
  "ignore": [
    "**/node_modules/**",
    "**/.git/**",
    "**/*.md",
    "**/myFolderToSkip/**"
  ]
}

jscpd documentation

jscpd - GitHub

Configuration in MegaLinter

Variable Description Default value
COPYPASTE_JSCPD_ARGUMENTS User custom arguments to add in linter CLI call
Ex: -s --foo "bar"
COPYPASTE_JSCPD_COMMAND_REMOVE_ARGUMENTS User custom arguments to remove from command line before calling the linter
Ex: -s --foo "bar"
COPYPASTE_JSCPD_FILTER_REGEX_INCLUDE Custom regex including filter
Ex: (src\|lib)
⚠️ Not available with COPYPASTE_JSCPD_CLI_LINT_MODE = project
Exclude no file
COPYPASTE_JSCPD_FILTER_REGEX_EXCLUDE Custom regex excluding filter
Ex: (test\|examples)
⚠️ Not available with COPYPASTE_JSCPD_CLI_LINT_MODE = project
Exclude no file
COPYPASTE_JSCPD_CLI_LINT_MODE Override default CLI lint mode
- project: Call the linter from the root of the project
project
COPYPASTE_JSCPD_FORWARD_EXCLUDED_DIRECTORIES In project CLI lint mode, forward excluded directories (EXCLUDED_DIRECTORIES + ADDITIONAL_EXCLUDED_DIRECTORIES) to the linter through its native exclusion arguments or generated ignore/config files true
COPYPASTE_JSCPD_PRE_COMMANDS List of bash commands to run before the linter None
COPYPASTE_JSCPD_POST_COMMANDS List of bash commands to run after the linter None
COPYPASTE_JSCPD_UNSECURED_ENV_VARIABLES List of env variables explicitly not filtered before calling COPYPASTE_JSCPD and its pre/post commands None
COPYPASTE_JSCPD_CONFIG_FILE jscpd configuration file name
Use LINTER_DEFAULT to let the linter find it
.jscpd.json
COPYPASTE_JSCPD_RULES_PATH Path where to find linter configuration file Workspace folder, then MegaLinter default rules
COPYPASTE_JSCPD_DISABLE_ERRORS Run linter but consider errors as warnings false
COPYPASTE_JSCPD_DISABLE_ERRORS_IF_LESS_THAN Maximum number of errors allowed 0
COPYPASTE_JSCPD_TIMEOUT_SECONDS Maximum duration in seconds of the linter run, after which the linter process and its child processes are killed and reported as an error (exit code 124). Overrides LINTER_TIMEOUT_SECONDS. 0 disables the timeout 300
COPYPASTE_JSCPD_CLI_EXECUTABLE Override CLI executable ['jscpd']

MegaLinter Flavors

This linter is available in the following flavors

Flavor Description Embedded linters Info
all Default MegaLinter Flavor 133 Docker Image Size (tag) Docker Pulls
c_cpp Optimized for pure C/C++ projects 59 Docker Image Size (tag) Docker Pulls
ci_light Optimized for CI items (Dockerfile, Jenkinsfile, JSON/YAML schemas,XML 23 Docker Image Size (tag) Docker Pulls
cupcake MegaLinter for the most commonly used languages 99 Docker Image Size (tag) Docker Pulls
documentation MegaLinter for documentation projects 52 Docker Image Size (tag) Docker Pulls
dotnet Optimized for C, C++, C# or VB based projects 66 Docker Image Size (tag) Docker Pulls
dotnetweb Optimized for C, C++, C# or VB based projects with JS/TS 81 Docker Image Size (tag) Docker Pulls
go Optimized for GO based projects 54 Docker Image Size (tag) Docker Pulls
java Optimized for JAVA based projects 57 Docker Image Size (tag) Docker Pulls
javascript Optimized for JAVASCRIPT or TYPESCRIPT based projects 68 Docker Image Size (tag) Docker Pulls
php Optimized for PHP based projects 57 Docker Image Size (tag) Docker Pulls
python Optimized for PYTHON based projects 69 Docker Image Size (tag) Docker Pulls
ruby Optimized for RUBY based projects 53 Docker Image Size (tag) Docker Pulls
rust Optimized for RUST based projects 53 Docker Image Size (tag) Docker Pulls
salesforce Optimized for Salesforce based projects 58 Docker Image Size (tag) Docker Pulls
swift Optimized for SWIFT based projects 53 Docker Image Size (tag) Docker Pulls
terraform Optimized for TERRAFORM based projects 57 Docker Image Size (tag) Docker Pulls

Behind the scenes

How are identified applicable files

  • If this linter is active, all files will always be linted

How the linting is performed

jscpd is called once on the whole project directory (project CLI lint mode)

  • filtering can not be done using MegaLinter configuration variables,it must be done using jscpd configuration or ignore file (if existing)
  • VALIDATE_ALL_CODEBASE: false doesn't make jscpd analyze only updated files

Example calls

jscpd --output ./report/copy-paste/ .
jscpd --output ./report/copy-paste/ -c .jscpd.json .

Help content

Copy/Paste Detector — find duplicated code

Usage: jscpd [OPTIONS] [PATH]...

Arguments:
  [PATH]...  Paths to scan for duplicates

Options:
  -k, --min-tokens <MIN_TOKENS>
          Minimum number of tokens to consider a duplicate
  -l, --min-lines <MIN_LINES>
          Minimum number of lines to consider a duplicate
  -x, --max-lines <MAX_LINES>
          Maximum number of lines per block to consider
      --max-gap-lines <N>
          Merge clones of the same file pair that are separated by at most N unmatched lines in both files into one near-miss clone (Type-3, reported as "similar"); 0 disables
      --similarity <RATIO>
          Report JavaScript/TypeScript function pairs whose AST similarity reaches RATIO as near-miss clones (Type-3, "similar"). A number in (0, 1]; the default 1 means exact matches only, e.g. 0.85 enables it
      --semantic
          Find semantic clones (Type-4, experimental): functions that do the same thing written differently, in one language or across languages, e.g. a Rust backend and a Svelte frontend. Compares embeddings of the functions' code, computed on this machine by a model that --semantic-download fetches once, or by an embeddings API (--semantic-url). Functions of JavaScript, TypeScript, JSX, TSX, Vue, Svelte, Astro, Python, Rust, Go, Java, Kotlin, C#, C, C++, PHP, Ruby, Scala and Swift
      --semantic-scope <SCOPE>
          Which semantic clones to report: all (default), same (within one language: several implementations of one feature) or cross (across languages: a rule written once per side) [possible values: all, same, cross]
      --semantic-provider <PROVIDER>
          Where embeddings come from: local (default: a model run in-process, fetched once by --semantic-download) or http (an OpenAI-compatible embeddings API at --semantic-url; giving a URL selects it) [possible values: local, http]
      --semantic-download [<MODEL>]
          Download a local embedding model into the jscpd cache directory, checking its checksum: MODEL, or the one --semantic-model names, or CodeRankEmbed (548 MB from huggingface.co). Alone it exits after the download; with --semantic it goes on to scan, with MODEL unless --semantic-model names another
      --semantic-rebuild-cache
          With --semantic: embed every function again and replace the cached vectors of the model in use for the scanned paths, instead of reusing them. The caches of other paths and models and the downloaded model stay
      --semantic-threshold <RATIO>
          Lowest cosine similarity of a semantic clone across languages, in (0, 1] (default: the model's calibrated value, 0.4125 for the default model; --semantic-models lists them, and a model not listed gets 0.6). A pair within one language needs more: see --semantic-same-threshold
      --semantic-same-threshold <RATIO>
          Lowest cosine similarity of a semantic clone within one language, in (0, 1] (default: the model's calibrated value, 0.6375 for the default model; when --semantic-threshold is set, that value plus the model's gap between the two, 0.225 for the default model and 0.15 for a model not listed by --semantic-models)
      --semantic-model <NAME>
          Embedding model for --semantic (default: CodeRankEmbed for the local provider; for http, unclemusclez/jina-embeddings-v2-base-code, Ollama's name for jina-embeddings-v2-base-code). A model that --semantic-models lists, named as it is there, by its Hugging Face id or by its Ollama name, gets its calibrated thresholds; an API gets the name as given
      --semantic-models
          List the embedding models jscpd has calibrated thresholds for, with their licenses and where they run, and exit
      --semantic-url <URL>
          OpenAI-compatible embeddings API for --semantic, e.g. http://localhost:11434/v1 for Ollama; selects the http provider. A key the API needs is read from the JSCPD_SEMANTIC_API_KEY environment variable, and is sent only to a URL given here or to a server on this machine
      --kind <LIST>
          Report only clones of these kinds: exact, renamed, similar, gap, ast, semantic (comma-separated). renamed needs --ignore-identifiers, --ignore-literals or --ignore-annotations; gap needs --max-gap-lines; ast needs --similarity; semantic needs --semantic
  -m, --mode <MODE>
          Detection mode: mild, weak, strict
      --skip-comments
          Alias for --mode weak (skip comment tokens)
  -f, --format <FORMAT>
          List of file extensions/formats to check (comma-separated)
  -i, --ignore <IGNORE>
          File-level glob patterns to ignore, e.g. "**/node_modules/**" (comma-separated)
      --ignore-pattern <IGNORE_PATTERN>
          Code-level regex patterns to skip matching tokens during detection, e.g. "//\\s*cpd-disable" (comma-separated)
  -r, --reporters <REPORTERS>
          Output reporters (comma-separated): console,json,xml,csv,html,markdown,badge,sarif,codeclimate,openmetrics,ai,xcode,threshold,silent,console-full Aliases: "full" and "consoleFull" are accepted for "console-full"; "gitlab" for "codeclimate"
  -o, --output <OUTPUT>
          Output directory for file reporters
  -c, --config <CONFIG>
          Path to config file (.jscpd.json)
      --exit-code [<EXIT_CODE>]
          Exit with code if duplicates found (default code: 1)
  -t, --threshold <THRESHOLD>
          Maximum duplication percentage before exit 1
      --sarif-error-tokens <TOKENS>
          Report SARIF results as "error" for clones with at least this many tokens (default: all "warning")
      --baseline <FILE>
          Path to a clone baseline file (e.g. .jscpd-baseline.json): clones whose fingerprint is absent from it are reported as new
      --update-baseline
          Rewrite the baseline file from the current run, creating it if missing, and print added/removed fingerprint counts (requires --baseline)
      --fail-on-new-clones [<N>]
          Exit 1 when more than N new clones are found (default N: 0; requires --baseline or --baseline-from-ref)
      --fail-on-empty
          Exit 1 when the scan analyzes no files: the paths exist but nothing matched the --format, --ignore and --pattern filters, or every file was below --min-tokens
      --baseline-from-ref <REF>
          Compare against an ephemeral baseline built from a git ref's tree (e.g. origin/main): the base ref is scanned with the same configuration and clones absent from it are reported as new
  -b, --blame
          Enrich clones with git blame data
      --no-gitignore
          Do not respect .gitignore files
      --follow-symlinks
          Follow symbolic links
  -z, --max-size <MAX_SIZE>
          Skip files larger than SIZE (e.g. 1kb, 1mb, 100kb, or raw bytes). Default: 1mb
      --workers <WORKERS>
          Number of worker threads (default: auto)
      --no-colors
          Disable ANSI color output
  -a, --absolute
          Use absolute paths in reports
      --ignore-case
          Ignore case of symbols in code (experimental)
      --ignore-identifiers
          Treat all identifiers as equal, so clones that differ only in variable, function or type names are found (Type-2 clones)
      --ignore-literals
          Treat all string and numeric literals as equal, so clones that differ only in literal values are found
      --ignore-annotations
          Skip annotations and decorators (@Name, @Name(...)) in Java, Kotlin, Scala, Groovy, Python, Dart, Swift, JavaScript and TypeScript
      --formats-exts <FORMATS_EXTS>
          Custom format-to-extension mappings (e.g. javascript:es,es6;dart:dt)
      --formats-names <FORMATS_NAMES>
          Custom format-to-filename mappings (e.g. makefile:Makefile,GNUmakefile;docker:Dockerfile)
      --cross-formats <CROSS_FORMATS>
          Detect clones across formats: semicolon-separated groups of comma-separated formats (e.g. "javascript,typescript;css,scss"). Preset: js-ts = javascript,jsx,typescript,tsx. TypeScript files in a group that also contains JavaScript are compared with type annotations stripped
  -p, --pattern <PATTERN>
          Glob pattern to find files to scan (e.g. **/*.ts, **/*.{js,ts})
      --list
          List all supported formats and exit
      --skip-local
          Skip clones where both fragments are in the same directory [alias: --skipLocal]
      --skip-isolated <GROUPS>
          Skip clones between different folders of the same isolation group: comma-separated groups of pipe-separated folders (e.g. "packages/a|packages/b,libs/a|libs/b") [alias: --skipIsolated]
      --mcp
          Serve the Model Context Protocol over stdio: scan PATHs once, then expose check_duplication / get_statistics / check_current_directory tools to MCP clients
      --lsp
          Serve the Language Server Protocol over stdio: an editor starts jscpd for its workspace and gets findings as diagnostics in the files it edits, updated as the text changes. Clones by default; --lsp-analyses picks the analyses. Each .jscpd.json in the workspace is a project of its own
      --lsp-analyses <LIST>
          The analyses --lsp runs, comma-separated: clones, ast (similar functions), semantic, dead-code, complexity, or all (default: clones). The lsp section of .jscpd.json and the editor's settings switch each on or off over this list
      --summary
          Print a codebase summary: top files and folders by tokens, lines, size, complexity
      --summary-top <N>
          Number of entries in each summary top list (default: 10)
      --summary-by <METRIC>
          Summary sort metric: tokens, lines, size, complexity (default: tokens)
      --history <RANGE>
          Duplication trend over git history: scan every commit in RANGE (e.g. v5.0.0..HEAD) with this configuration and print a chart and a table
      --history-since <DATE>
          Like --history, selecting commits since DATE (e.g. 2026-01-01); combines with --history to bound the range
      --history-every <N>
          Keep every Nth commit of the history series, counted from the newest (default: 1)
      --history-limit <N>
          Maximum number of commits in the history series, sampled evenly (default: 30)
      --dead-code
          Find dead code instead of duplicates: unused files, exports, symbols and imports across JavaScript, TypeScript and Python
      --compare
          Compare two folders function by function: which functions of the first have a counterpart in the second and which do not, and the other way round. For a port to another language or platform, the source first and the target second (jscpd --compare python-lib/ rust-lib/), or two implementations of one app (jscpd --compare ios/ android/). Pairs functions with the --semantic model, so the --semantic-* options apply; counts functions of at least --min-lines lines and --min-tokens tokens (30 by default here). Reporters: console, console-full, json, markdown
      --complexity
          Report complexity only: the --summary tables ranked by complexity, without clone detection (reporters: console, ai, json)
      --dashboard
          Print one screen with the whole picture: the health score, project size, duplication, complexity and dead code (JavaScript, TypeScript, Python). Reporters: console, json, badge, markdown, html
      --health
          Print only the project health badge: one 0-100 score with a grade, from duplication, dead code and complexity, plus the metrics of --health-input. Reporters: console, ai, json, badge, markdown, html
      --health-input <FILE>
          JSON file with metrics from other tools (coverage, tests, security) to include in the health score: {"metrics": [{"id", "score"} or {"id", "value", "halfLife", "direction"}]}
      --dead-code-categories <LIST>
          Dead-code findings to report: unused-file, unused-export, unused-symbol, unused-import, unused-member, or `all` (with --dead-code)
      --min-confidence <N>
          Drop dead-code findings below this confidence, 0-100 (with --dead-code)
      --rust-diagnostics <FILE>
          Rust dead code from the compiler, for --dead-code, --dashboard and --health: a file of `cargo check --message-format=json` output, or `-` to read it from stdin
      --entry <GLOB>
          Treat files matching this glob as dead-code entry points (repeatable)
      --include-tests
          Report dead code inside test, fixture and example files
      --include-entry-exports
          Report exports of entry-point files, which are usually a public API
  -s, --silent
          Do not write detection progress and result to console
      --no-tips
          Do not print tips and promotional messages after detection (also skipped when stdout is not a terminal or CI or JSCPD_NO_TIPS is set)
      --debug
          Print merged config (CLI + config file) as JSON and exit without running detection
  -h, --help
          Print help
  -V, --version
          Print version

Installation on mega-linter Docker image

  • Dockerfile commands :
    # renovate: datasource=npm depName=jscpd
    ARG NPM_JSCPD_VERSION=5.4.0
    

Comments