AST-Based Code Fingerprinting

AST hashing produces format-independent fingerprints for Python source code by operating on the Abstract Syntax Tree rather than raw text. Two files with identical logic but different whitespace, comments, or variable names can produce the same fingerprint, making this ideal for code plagiarism detection, license compliance, and structural change tracking.

How It Works

1. The source code is parsed into a Python AST.
2. The AST is normalized according to the chosen level (strip comments, rename variables, normalize literals).
3. A structural hash is computed from the normalized tree.
4. A node-type frequency signature enables cosine-similarity comparison even when hashes differ.

Quick Start

from primestamp.fingerprint.ast_hash import ASTHashGenerator, NormalizationLevel

gen = ASTHashGenerator(normalization=NormalizationLevel.COMMENTS)

fp1 = gen.generate(b"def hello():\n    print('hi')")
fp2 = gen.generate(b"def hello():\n    # greeting\n    print('hi')")

# Comments are stripped -- structural hash matches
assert fp1.structural_hash == fp2.structural_hash

Normalization Levels

Level	Strips Comments	Normalizes Names	Normalizes Literals
NONE	No	No	No
COMMENTS	Yes	No	No
NAMES	Yes	Yes	No
FULL	Yes	Yes	Yes

# Full normalization: variable names and literals are canonicalized
gen_full = ASTHashGenerator(normalization=NormalizationLevel.FULL)

code_a = b"def add(x, y):\n    return x + y"
code_b = b"def sum_values(a, b):\n    return a + b"

fp_a = gen_full.generate(code_a)
fp_b = gen_full.generate(code_b)

# With FULL normalization, these are structurally identical
assert fp_a.structural_hash == fp_b.structural_hash

Structural Similarity

When two files are not identical but share structure, use cosine similarity on the node-type frequency signature.

similarity = fp1.structural_similarity(fp2)
print(f"Structural similarity: {similarity:.4f}")  # 0.0 to 1.0

Function-Level Hashing

Extract per-function fingerprints for fine-grained comparison.

gen = ASTHashGenerator(include_functions=True, include_classes=True)
fp = gen.generate(source_bytes)

for func in fp.function_hashes:
    print(f"{func['name']} (line {func['line']}): {func['hash'][:16]}...")
    print(f"  args: {func['arg_count']}, body size: {func['body_size']}")

for cls in fp.class_hashes:
    print(f"class {cls['name']}: {cls['method_count']} methods")

Configuration

Parameter	Type	Default	Description
normalization	NormalizationLevel	COMMENTS	How aggressively to normalize the AST.
include_functions	bool	True	Extract per-function hashes.
include_classes	bool	True	Extract per-class hashes.

Fingerprint Fields

Field	Description
structural_hash	SHA-256 of the normalized AST structure.
node_signature	Dict of AST node type counts (e.g., {"FunctionDef": 3, "If": 5}).
function_count	Number of functions detected.
class_count	Number of classes detected.
import_count	Number of import statements.
line_count	Source line count.
source_hash	SHA-256 of the raw source bytes.

Factory Function

from primestamp.fingerprint.ast_hash import create_ast_hash_generator

gen = create_ast_hash_generator(
    normalization="full",
    include_functions=True,
    include_classes=True,
)

Language Support

AST hashing currently supports Python source code only. The module uses the built-in ast module and requires valid Python syntax. Files with syntax errors receive a fallback SHA-256 hash of the raw source.

Use Cases

• Code plagiarism detection: Use NAMES or FULL normalization to catch renamed copies.
• Change tracking in CI/CD: Compare structural hashes between commits to detect logic changes vs. formatting changes.
• License compliance: Fingerprint open-source code and match against your codebase.