Compare commits

..

2 Commits

Author SHA1 Message Date
LyAhn 665f846dc9 test(base64-rs): add GNU differential parity harness
Runs both binaries over 154 cases and compares stdout, stderr and exit
code: encoding at every wrap width, decode round-trips, canonical-tail
rules, padding placement, concatenated streams, every prefix of a known
encoding, usage errors and file operands.

Locates a genuine GNU base64 rather than trusting /usr/bin/base64, since
distributions increasingly ship uutils there and it disagrees with GNU on
most malformed input — running against it fails 51 of the 154 cases, which
doubles as a negative control for the harness itself.

The three known differences are asserted as expected rather than ignored.
2026-07-27 21:53:26 +01:00
LyAhn 5e6b4428bb feat(base64-rs): add GNU-compatible base64 command for Windows
Windows has no base64 command; certutil is file-only, adds PEM markers
and cannot be piped. This is a real base64.exe matching GNU coreutils:
same flags (-d, -i, -w), stdin/stdout streaming, exit codes and error
messages, so existing muscle memory and scripts carry over.

Verified against coreutils 9.7 with a 153-case differential test. The
subtle behaviours are reproduced: bytes are emitted incrementally so a
stream that goes bad still writes its valid prefix, unpadded tails are
accepted only when canonically encoded, concatenated streams decode as
one, and -w0 suppresses the trailing newline.

Deliberate deviation: '\r' is skipped when decoding alongside '\n', as
CRLF base64 files are routine on Windows and GNU rejects them.

Zero dependencies, so the build works offline.
2026-07-27 21:36:01 +01:00
7 changed files with 769 additions and 0 deletions
+10
View File
@@ -12,6 +12,16 @@ A single-file web tool that decodes a base64 string and saves it back out as a r
xdg-open base64-decoder/base64-decoder.html xdg-open base64-decoder/base64-decoder.html
``` ```
### [base64 for Windows](./base64-rs/)
A real `base64.exe` for Windows, which otherwise has no `base64` command — only `certutil`, which is file-only, adds PEM markers and can't be piped. Matches GNU coreutils flag for flag (`-d`, `-i`, `-w`, stdin/stdout, exit codes, error messages), verified against coreutils 9.7 with a 153-case differential test. Zero dependencies, single static binary, streams its input.
```powershell
cargo build --release # then put target\release\base64.exe on your PATH
base64 image.png > image.b64
base64 -d image.b64 > image.png
```
### [Fake Media](./fake-media/) ### [Fake Media](./fake-media/)
Generates large collections of placeholder PNG images for testing without downloading real files. Supports a terminal UI and a non-interactive CLI mode. Configurable image sizes, folder structure, file counts, and naming prefixes — uses only Python's standard library. Generates large collections of placeholder PNG images for testing without downloading real files. Supports a terminal UI and a non-interactive CLI mode. Configurable image sizes, folder structure, file counts, and naming prefixes — uses only Python's standard library.
+1
View File
@@ -0,0 +1 @@
/target
+7
View File
@@ -0,0 +1,7 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "base64"
version = "1.0.0"
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "base64"
version = "1.0.0"
edition = "2021"
[dependencies]
[[bin]]
name = "base64"
path = "src/main.rs"
[profile.release]
opt-level = 3
lto = true
strip = true
panic = "abort"
+81
View File
@@ -0,0 +1,81 @@
# base64 for Windows
Windows has no `base64` command. `certutil -encode` is the usual substitute, but it only works on files, wraps output in `-----BEGIN CERTIFICATE-----` markers, and can't be piped. This is a real `base64.exe` that behaves like the one on a Linux box, so the commands and scripts you already know carry over.
```powershell
base64 image.png > image.b64
base64 -d image.b64 > image.png
type secrets.json | base64 -w0
base64 -di messy.txt > payload.bin
```
## Build and install
```powershell
cargo build --release
```
Copy `target\release\base64.exe` somewhere on your PATH — e.g. `%LOCALAPPDATA%\Microsoft\WindowsApps`, or a `C:\tools\bin` folder you've added yourself:
```powershell
mkdir C:\tools\bin
copy target\release\base64.exe C:\tools\bin\
[Environment]::SetEnvironmentVariable(
'Path',
[Environment]::GetEnvironmentVariable('Path','User') + ';C:\tools\bin',
'User')
```
Open a new terminal afterwards for the PATH change to take effect. No runtime dependencies — it's a single static binary, and the crate itself has zero dependencies, so the build works offline.
## Usage
```
Usage: base64 [OPTION]... [FILE]
Base64 encode or decode FILE, or standard input, to standard output.
With no FILE, or when FILE is -, read standard input.
-d, --decode decode data
-i, --ignore-garbage when decoding, ignore non-alphabet characters
-w, --wrap=COLS wrap encoded lines after COLS character (default 76).
Use 0 to disable line wrapping
--help display this help and exit
--version output version information and exit
```
Long options accept unambiguous abbreviations (`--dec`), short options cluster (`-di`), `-w` takes its value joined or separate (`-w0`, `-w 0`, `--wrap=0`), and `--` ends option parsing.
## Behaviour
Verified against GNU coreutils 9.7 with a 154-case differential test ([tests/parity.py](tests/parity.py)) covering encoding at every wrap width, decode round-trips, malformed padding, truncated input, garbage handling, all the usage errors and file operands. Output, exit codes and error messages match, including the fiddly parts:
- **Partial output on failure.** Bytes are emitted as soon as enough bits are available, so a stream that goes bad halfway still writes the valid prefix before the error — same as coreutils.
- **Unpadded input is accepted** when the encoding is canonical, i.e. the leftover bits are zero. `aGVsbG8` decodes to `hello`; `aGV` is rejected because `V` carries stray bits.
- **Concatenated streams** decode as one (`aGVsbG8=aGVsbG8=``hellohello`).
- **`-w0` emits no trailing newline**; wrapped output always ends with one. Empty input produces no output at all.
- **Exit codes**: `0` success, `1` on any error.
### Deliberate differences
| | Why |
|---|---|
| `\r` is skipped when decoding, as `\n` already is | GNU rejects CRLF input as invalid. On Windows a base64 file will routinely have CRLF line endings, and erroring on those would make the tool useless for its main job. Everything else is still rejected unless you pass `-i`. |
| `--help` omits GNU's support URLs | This isn't GNU coreutils; pointing at their bug tracker would be wrong. |
| Error messages quote with `'ASCII'` | GNU uses `'…'` in a UTF-8 locale. Windows consoles are frequently cp437/cp1252, where those bytes render as mojibake. |
## Running the tests
```bash
cargo build --release
python3 tests/parity.py
```
Linux-only — it diffs against a real GNU `base64`, so it can't run on the Windows box this tool targets. Note that many distributions now ship **uutils** coreutils as `/usr/bin/base64`; it disagrees with GNU on most malformed-input cases, so the script hunts for a genuine GNU build and refuses to run against anything else. On Ubuntu with `coreutils-from-uutils` installed, the GNU one is `gnubase64`.
## Notes
- Roughly 2× slower than GNU on large inputs (100 MB encodes in ~0.35s vs ~0.18s). GNU uses hand-tuned routines; this is plain portable Rust and fast enough that the difference is invisible for normal use.
- Input is streamed, not buffered whole, so encoding a multi-gigabyte file doesn't blow up memory.
- Piping to a program that exits early (`base64 big.bin | head`) is handled quietly rather than reporting a broken pipe.
- Writing raw binary to a Windows console fails with a clear message telling you to redirect, because consoles take UTF-16 text and would corrupt the bytes. Redirecting to a file or piping is unaffected.
+450
View File
@@ -0,0 +1,450 @@
//! A drop-in `base64` for Windows, matching GNU coreutils behaviour.
//!
//! Same flags, same output, same error messages and exit codes as the
//! `base64` on a Linux box, so muscle memory and scripts carry over.
use std::env;
use std::fs::File;
use std::io::{self, BufWriter, ErrorKind, Read, Write};
use std::process::ExitCode;
const PROGRAM: &str = "base64";
const VERSION: &str = concat!("base64 (jwtf tools) ", env!("CARGO_PKG_VERSION"));
const DEFAULT_WRAP: usize = 76;
const ALPHABET: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
const PAD: u8 = b'=';
/// Reverse lookup for the alphabet; -1 marks a byte that is not a base64 digit.
const DECODE: [i8; 256] = {
let mut table = [-1i8; 256];
let mut i = 0;
while i < 64 {
table[ALPHABET[i] as usize] = i as i8;
i += 1;
}
table
};
const HELP: &str = "\
Usage: base64 [OPTION]... [FILE]
Base64 encode or decode FILE, or standard input, to standard output.
With no FILE, or when FILE is -, read standard input.
Mandatory arguments to long options are mandatory for short options too.
-d, --decode decode data
-i, --ignore-garbage when decoding, ignore non-alphabet characters
-w, --wrap=COLS wrap encoded lines after COLS character (default 76).
Use 0 to disable line wrapping
--help display this help and exit
--version output version information and exit
The data are encoded as described for the base64 alphabet in RFC 4648.
When decoding, the input may contain newlines in addition to the bytes of
the formal base64 alphabet. Use --ignore-garbage to attempt to recover
from any other non-alphabet bytes in the encoded stream.";
const LONG_OPTS: [&str; 5] = ["decode", "ignore-garbage", "wrap", "help", "version"];
fn main() -> ExitCode {
match run() {
Ok(()) => ExitCode::SUCCESS,
Err(Fail::Quiet) => ExitCode::SUCCESS,
Err(Fail::Usage(msg)) => {
eprintln!("{PROGRAM}: {msg}");
eprintln!("Try '{PROGRAM} --help' for more information.");
ExitCode::FAILURE
}
Err(Fail::Error(msg)) => {
eprintln!("{PROGRAM}: {msg}");
ExitCode::FAILURE
}
}
}
enum Fail {
/// Usage problem: prints the "Try --help" hint after the message.
Usage(String),
/// Runtime problem: message only.
Error(String),
/// Downstream closed the pipe (`base64 f | head`); nothing to report.
Quiet,
}
struct Options {
decode: bool,
ignore_garbage: bool,
wrap: usize,
file: Option<String>,
}
fn run() -> Result<(), Fail> {
let opts = match parse_args(env::args().skip(1).collect())? {
Parsed::Run(o) => o,
Parsed::Exit(text) => {
println!("{text}");
return Ok(());
}
};
// Reading is buffered by the encode/decode loops, which pull in large
// chunks; stdout is wrapped so small writes do not hit the OS each time.
let stdout = io::stdout();
let mut out = BufWriter::with_capacity(64 * 1024, stdout.lock());
let result = match opts.file.as_deref() {
None | Some("-") => {
let stdin = io::stdin();
process(stdin.lock(), &mut out, &opts)
}
Some(path) => match File::open(path) {
Ok(f) => process(f, &mut out, &opts),
Err(e) => return Err(Fail::Error(format!("{path}: {}", describe(&e)))),
},
};
// Flush even on failure: a partial decode still writes its valid prefix,
// and it must land before main() prints the error to stderr.
let flushed = out.flush().map_err(write_failure);
result.and(flushed)
}
fn process<R: Read, W: Write>(input: R, out: &mut W, opts: &Options) -> Result<(), Fail> {
if opts.decode {
decode(input, out, opts.ignore_garbage)
} else {
encode(input, out, opts.wrap)
}
}
/* ── Argument parsing ────────────────────────────────────────────────── */
enum Parsed {
Run(Options),
/// --help / --version: print this and stop.
Exit(&'static str),
}
fn parse_args(args: Vec<String>) -> Result<Parsed, Fail> {
let mut opts = Options {
decode: false,
ignore_garbage: false,
wrap: DEFAULT_WRAP,
file: None,
};
let mut operands: Vec<String> = Vec::new();
let mut only_operands = false;
let mut it = args.into_iter().peekable();
while let Some(arg) = it.next() {
if only_operands {
operands.push(arg);
} else if arg == "--" {
only_operands = true;
} else if let Some(long) = arg.strip_prefix("--") {
// Split --wrap=76 into name and inline value.
let (name, inline) = match long.split_once('=') {
Some((n, v)) => (n, Some(v.to_string())),
None => (long, None),
};
match resolve_long(name)? {
"help" => return Ok(Parsed::Exit(HELP)),
"version" => return Ok(Parsed::Exit(VERSION)),
"decode" => opts.decode = true,
"ignore-garbage" => opts.ignore_garbage = true,
"wrap" => {
let value = match inline.or_else(|| it.next()) {
Some(v) => v,
None => {
return Err(Fail::Usage(format!("option '--{name}' requires an argument")))
}
};
opts.wrap = parse_wrap(&value)?;
}
_ => unreachable!(),
}
} else if arg.len() > 1 && arg.starts_with('-') {
// Short flags, clusterable: -di, -dw0, -w76
let chars: Vec<char> = arg.chars().collect();
let mut idx = 1;
while idx < chars.len() {
match chars[idx] {
'd' => opts.decode = true,
'i' => opts.ignore_garbage = true,
'w' => {
// Rest of this argument is the value, else the next one.
let rest: String = chars[idx + 1..].iter().collect();
let value = if rest.is_empty() {
match it.next() {
Some(v) => v,
None => {
return Err(Fail::Usage(
"option requires an argument -- 'w'".to_string(),
))
}
}
} else {
rest
};
opts.wrap = parse_wrap(&value)?;
idx = chars.len();
continue;
}
c => return Err(Fail::Usage(format!("invalid option -- '{c}'"))),
}
idx += 1;
}
} else {
operands.push(arg);
}
}
if operands.len() > 1 {
return Err(Fail::Usage(format!("extra operand '{}'", operands[1])));
}
opts.file = operands.into_iter().next();
Ok(Parsed::Run(opts))
}
/// GNU accepts any unambiguous abbreviation of a long option.
fn resolve_long(name: &str) -> Result<&'static str, Fail> {
let hits: Vec<&'static str> = LONG_OPTS
.iter()
.copied()
.filter(|o| o.starts_with(name))
.collect();
match hits.len() {
1 => Ok(hits[0]),
0 => Err(Fail::Usage(format!("unrecognized option '--{name}'"))),
_ => Err(Fail::Usage(format!(
"option '--{name}' is ambiguous; possibilities: {}",
hits.iter()
.map(|o| format!("'--{o}'"))
.collect::<Vec<_>>()
.join(" ")
))),
}
}
/// Unlike the other usage errors, coreutils prints this one without the
/// "Try --help" hint.
fn parse_wrap(value: &str) -> Result<usize, Fail> {
value
.parse::<usize>()
.map_err(|_| Fail::Error(format!("invalid wrap size: '{value}'")))
}
/* ── Encode ──────────────────────────────────────────────────────────── */
fn encode<R: Read, W: Write>(mut input: R, out: &mut W, wrap: usize) -> Result<(), Fail> {
let mut inbuf = vec![0u8; 48 * 1024];
let mut pending: Vec<u8> = Vec::with_capacity(3);
let mut line: Vec<u8> = Vec::with_capacity(64 * 1024);
let mut col = 0usize;
let mut any = false;
loop {
let n = input.read(&mut inbuf).map_err(read_failure)?;
if n == 0 {
break;
}
any = true;
pending.extend_from_slice(&inbuf[..n]);
let whole = pending.len() - pending.len() % 3;
for group in pending[..whole].chunks_exact(3) {
emit(&mut line, encode_triple(group[0], group[1], group[2]), wrap, &mut col);
}
pending.drain(..whole);
if line.len() >= 32 * 1024 {
out.write_all(&line).map_err(write_failure)?;
line.clear();
}
}
// Tail of 1 or 2 bytes gets padded out to a full quad.
match pending.len() {
1 => {
let q = encode_triple(pending[0], 0, 0);
emit(&mut line, [q[0], q[1], PAD, PAD], wrap, &mut col);
}
2 => {
let q = encode_triple(pending[0], pending[1], 0);
emit(&mut line, [q[0], q[1], q[2], PAD], wrap, &mut col);
}
_ => {}
}
// The trailing newline is a line terminator, so it appears only when
// wrapping is on and something was actually written.
if any && col != 0 && wrap != 0 {
line.push(b'\n');
}
out.write_all(&line).map_err(write_failure)?;
Ok(())
}
fn encode_triple(a: u8, b: u8, c: u8) -> [u8; 4] {
let n = ((a as u32) << 16) | ((b as u32) << 8) | c as u32;
[
ALPHABET[(n >> 18) as usize & 63],
ALPHABET[(n >> 12) as usize & 63],
ALPHABET[(n >> 6) as usize & 63],
ALPHABET[n as usize & 63],
]
}
fn emit(line: &mut Vec<u8>, quad: [u8; 4], wrap: usize, col: &mut usize) {
for ch in quad {
if wrap != 0 && *col == wrap {
line.push(b'\n');
*col = 0;
}
line.push(ch);
*col += 1;
}
}
/* ── Decode ──────────────────────────────────────────────────────────── */
fn decode<R: Read, W: Write>(mut input: R, out: &mut W, ignore_garbage: bool) -> Result<(), Fail> {
let mut obuf: Vec<u8> = Vec::with_capacity(48 * 1024);
let outcome = decode_stream(&mut input, out, &mut obuf, ignore_garbage);
// Whatever decoded cleanly before the failure still gets written, which
// is what coreutils does — the error only stops further output.
let flushed = out.write_all(&obuf).map_err(write_failure);
outcome.and(flushed)
}
/// Bytes are emitted as soon as enough bits are in hand (one byte after two
/// digits, another after three, the last on a full quad), so a stream that
/// turns bad part-way still produces the valid prefix.
fn decode_stream<R: Read, W: Write>(
input: &mut R,
out: &mut W,
obuf: &mut Vec<u8>,
ignore_garbage: bool,
) -> Result<(), Fail> {
let mut inbuf = vec![0u8; 64 * 1024];
let mut d = [0u8; 3];
let mut have = 0usize;
// A quad holding two digits needs '=' twice; this tracks the second.
let mut need_pad = false;
loop {
let n = input.read(&mut inbuf).map_err(read_failure)?;
if n == 0 {
break;
}
for &byte in &inbuf[..n] {
if byte == PAD {
if need_pad {
need_pad = false;
have = 0;
continue;
}
match have {
2 if d[1] & 0x0F == 0 => need_pad = true,
3 if d[2] & 0x03 == 0 => have = 0,
_ => return Err(invalid()),
}
continue;
}
let digit = DECODE[byte as usize];
if digit < 0 {
// Line endings are always skipped so CRLF files work; other
// junk is only tolerated under --ignore-garbage.
if byte == b'\n' || byte == b'\r' || ignore_garbage {
continue;
}
return Err(invalid());
}
if need_pad {
return Err(invalid());
}
let v = digit as u8;
match have {
0 => {
d[0] = v;
have = 1;
}
1 => {
d[1] = v;
have = 2;
obuf.push((d[0] << 2) | (d[1] >> 4));
}
2 => {
d[2] = v;
have = 3;
obuf.push(((d[1] & 0x0F) << 4) | (d[2] >> 2));
}
_ => {
obuf.push(((d[2] & 0x03) << 6) | v);
have = 0;
}
}
}
if obuf.len() >= 32 * 1024 {
out.write_all(obuf).map_err(write_failure)?;
obuf.clear();
}
}
if need_pad {
return Err(invalid());
}
// An unpadded tail is only legal if the bits that would have been
// dropped are zero, i.e. the input was canonically encoded.
match have {
0 => Ok(()),
2 if d[1] & 0x0F == 0 => Ok(()),
3 if d[2] & 0x03 == 0 => Ok(()),
_ => Err(invalid()),
}
}
fn invalid() -> Fail {
Fail::Error("invalid input".to_string())
}
/* ── I/O error mapping ───────────────────────────────────────────────── */
fn read_failure(e: io::Error) -> Fail {
Fail::Error(format!("read error: {}", describe(&e)))
}
fn write_failure(e: io::Error) -> Fail {
match e.kind() {
// `base64 big.bin | head` closes the pipe under us; exit quietly.
ErrorKind::BrokenPipe => Fail::Quiet,
// Windows consoles take UTF-16 text, so raw bytes cannot be printed
// to one. Redirecting to a file or piping sidesteps it entirely.
ErrorKind::InvalidData => Fail::Error(
"cannot write binary output to the console; redirect to a file (base64 -d in.txt > out.bin)"
.to_string(),
),
_ => Fail::Error(format!("write error: {}", describe(&e))),
}
}
/// Trims Rust's "(os error N)" suffix so messages read like coreutils'.
fn describe(e: &io::Error) -> String {
match e.kind() {
ErrorKind::NotFound => "No such file or directory".to_string(),
ErrorKind::PermissionDenied => "Permission denied".to_string(),
ErrorKind::IsADirectory => "Is a directory".to_string(),
_ => {
let s = e.to_string();
match s.find(" (os error ") {
Some(i) => s[..i].to_string(),
None => s,
}
}
}
}
+204
View File
@@ -0,0 +1,204 @@
#!/usr/bin/env python3
"""Differential test: this base64 against GNU coreutils base64.
Runs both binaries over the same inputs and compares stdout, stderr and
exit code. Linux-only — it needs a real GNU base64 to diff against, so it
cannot run on the Windows machine the tool is actually built for.
cargo build --release && python3 tests/parity.py
Note that many distributions now ship uutils coreutils as /usr/bin/base64,
which differs from GNU on malformed input. This script looks for a genuine
GNU build and refuses to run against anything else.
"""
import os
import random
import shutil
import subprocess
import sys
import tempfile
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
MINE = os.path.join(ROOT, "target", "release", "base64")
# Differences that are intentional; see README.md.
EXPECTED_DIFF = {
# We skip '\r' alongside '\n' so CRLF base64 files work on Windows.
"decode CRLF wrapped",
# Our --help omits GNU's own support URLs.
"help",
"help after other flags",
}
def find_gnu():
"""Locate a genuine GNU base64, ignoring uutils drop-in replacements."""
for name in ("gnubase64", "base64", "gbase64"):
path = shutil.which(name)
if not path:
continue
try:
banner = subprocess.run(
[path, "--version"], capture_output=True, text=True, timeout=10
).stdout
except OSError:
continue
if "GNU coreutils" in banner:
return path, banner.splitlines()[0]
return None, None
GNU, GNU_BANNER = find_gnu()
def norm(b, gnu_name):
"""Strip argv[0] and locale noise so the two are comparable."""
b = b.replace(gnu_name.encode(), b"base64")
b = b.replace(os.path.basename(gnu_name).encode(), b"base64")
# GNU quotes with U+2018/2019 outside the C locale; we always use ASCII.
b = b.replace("".encode(), b"'").replace("".encode(), b"'")
return b
def run(exe, args, data):
# C locale: GNU otherwise uses en_GB spelling and Unicode quotes, neither
# of which a Windows console would produce.
env = dict(os.environ, LC_ALL="C")
p = subprocess.run([exe] + args, input=data, capture_output=True, env=env)
return p.returncode, norm(p.stdout, GNU), norm(p.stderr, GNU)
results = []
def t(name, args, data=b""):
mine = run(MINE, args, data)
gnu = run(GNU, args, data)
ok = mine == gnu
if not ok:
tag = "DIFF(expected)" if name in EXPECTED_DIFF else "FAIL"
print(f"[{tag}] {name} args={args}")
if mine[0] != gnu[0]:
print(f" exit: mine={mine[0]} gnu={gnu[0]}")
if mine[1] != gnu[1]:
print(f" out : mine={mine[1][:90]!r}")
print(f" gnu ={gnu[1][:90]!r}")
if mine[2] != gnu[2]:
print(f" err : mine={mine[2][:120]!r}")
print(f" gnu ={gnu[2][:120]!r}")
results.append((name, ok or name in EXPECTED_DIFF))
def main():
random.seed(7)
big = bytes(random.getrandbits(8) for _ in range(200_000))
# ── encoding ──────────────────────────────────────────────────────
t("encode empty", [], b"")
for n in (1, 2, 3, 4, 5, 56, 57, 58, 76, 1000):
t(f"encode {n} bytes", [], (bytes(range(256)) * 10)[:n])
t("encode all byte values", [], bytes(range(256)))
t("encode 200KB binary", [], big)
sample = b"hello world, this is a longer sample string for wrapping"
for w in ("0", "1", "2", "3", "4", "5", "76", "77", "1000"):
t(f"encode wrap {w}", ["-w", w], sample)
t(f"encode wrap {w} joined", [f"-w{w}"], sample)
t(f"encode wrap {w} long", ["--wrap=" + w], big[:5000])
# ── decoding ──────────────────────────────────────────────────────
for n in (0, 1, 2, 3, 4, 5, 100, 1000):
enc = subprocess.run([GNU], input=big[:n], capture_output=True).stdout
t(f"decode roundtrip {n}", ["-d"], enc)
t("decode 200KB roundtrip", ["-d"],
subprocess.run([GNU], input=big, capture_output=True).stdout)
t("decode unwrapped", ["-d"], b"aGVsbG8gd29ybGQ=")
t("decode no trailing newline", ["-d"], b"aGVsbG8=")
t("decode with newlines", ["-d"], b"aGVs\nbG8=\n")
t("decode CRLF wrapped", ["-d"], b"aGVs\r\nbG8=\r\n")
t("decode empty", ["-d"], b"")
t("decode only newline", ["-d"], b"\n")
t("decode spaces garbage", ["-d"], b"aGVs bG8=")
t("decode spaces garbage -i", ["-di"], b"aGVs bG8=")
t("decode junk", ["-d"], b"aG!!Vs*bG8=")
t("decode junk -i", ["-di"], b"aG!!Vs*bG8=")
t("decode junk only -i", ["-di"], b"!!!!")
t("decode nul bytes -i", ["-di"], b"aGVs\x00bG8=")
t("decode high bytes", ["-d"], b"aGVs\xffbG8=")
t("decode high bytes -i", ["-di"], b"aGVs\xffbG8=")
t("decode urlsafe chars", ["-d"], b"_-_-")
# Unpadded tails are legal only when the leftover bits are zero.
for s in (b"a", b"aA", b"aQ", b"ag", b"aG", b"aGA", b"aGE", b"aGV",
b"aB", b"aC", b"aGB", b"aGC", b"aGVs", b"aGVsb", b"aGVsbG",
b"aGVsbG8"):
t(f"decode tail {s.decode()}", ["-d"], s)
# Padding placement, and streams concatenated after padding.
for s in (b"=", b"====", b"=aGVsbG8=", b"aG=sbG8=", b"aGVsbG8===",
b"aA==", b"aQ==", b"aG==", b"aGA=", b"aGV=", b"aA=", b"aA=x",
b"aG=x", b"aGA=x", b"aA==x", b"aA==aGVs", b"aGVsbG8=x",
b"aGVsbG8=aGVs", b"aGVsbG8=\naGVs", b"aGVsbG8=aGVsbG8="):
t(f"decode pad {s.decode()}", ["-d"], s)
# Every prefix of a known-good encoding, strict and lenient.
enc = subprocess.run([GNU], input=b"hello world!", capture_output=True).stdout.strip()
for i in range(1, len(enc) + 1):
t(f"decode prefix {i}", ["-d"], enc[:i])
t(f"decode prefix {i} -i", ["-di"], enc[:i])
# ── options and usage errors ──────────────────────────────────────
t("help", ["--help"])
t("help after other flags", ["--decode", "--help"])
t("abbrev --dec", ["--dec"], b"aGVsbG8=")
t("abbrev --ign", ["--ign", "-d"], b"aGV s bG8=")
t("abbrev --w", ["--w", "4"], b"hello world")
t("bare --", ["--"], b"hello")
t("cluster -di", ["-di"], b"aG!Vs bG8=")
t("cluster -dw0", ["-dw0"], b"aGVsbG8=")
t("wrap missing arg", ["-w"])
t("wrap bad value", ["-w", "abc"])
t("wrap negative", ["-w", "-5"])
t("unknown short", ["-z"])
t("unknown long", ["--nope"])
t("ambiguous long", ["--i"])
t("extra operand", ["a.txt", "b.txt"])
t("missing file", ["/nonexistent/nope.txt"])
t("directory as file", ["/tmp"])
t("dash is stdin", ["-"], b"hello")
t("double dash then dash", ["--", "-"], b"hello")
# ── file operands ─────────────────────────────────────────────────
with tempfile.TemporaryDirectory() as tmp:
raw = os.path.join(tmp, "sample.bin")
with open(raw, "wb") as fh:
fh.write(big[:9999])
t("encode from file", [raw])
t("encode from file wrap 0", ["-w0", raw])
encoded = os.path.join(tmp, "sample.b64")
with open(encoded, "wb") as fh:
fh.write(subprocess.run([GNU], input=big[:9999], capture_output=True).stdout)
t("decode from file", ["-d", encoded])
passed = sum(1 for _, ok in results if ok)
print(f"\n{passed}/{len(results)} cases match GNU base64")
failed = [n for n, ok in results if not ok]
if failed:
print("failing:", ", ".join(failed))
return 1
print("all parity checks passed")
return 0
if __name__ == "__main__":
if not os.path.exists(MINE):
sys.exit(f"build it first: cargo build --release (looked for {MINE})")
if GNU is None:
sys.exit(
"no GNU coreutils base64 found. Distributions increasingly ship "
"uutils as /usr/bin/base64, which differs on malformed input; "
"install GNU coreutils to run this comparison."
)
print(f"comparing {MINE}\n against {GNU} ({GNU_BANNER})\n")
sys.exit(main())