SnowLuma v2.4.2 source snapshot
Dev Build / cleanup (push) Waiting to run
Dev Build / typecheck (push) Waiting to run
Publish MCP to npm / Build, test, and publish MCP (push) Waiting to run
Publish SDK to npm / Build, test, and publish SDK (push) Waiting to run
Sync action catalog to docs / sync (push) Waiting to run
Publish UI to npm / Build and publish UI (push) Waiting to run
Build and Release / preflight (push) Waiting to run

This commit is contained in:
TanTan committed 2026-11-13 09:02:00 +08:00
commit fa9c73363d
1367 files changed
+355735

No files matched your search

+131
View File
@@ -0,0 +1,131 @@
# @snowluma/proton
Compile-time protobuf codecs for SnowLuma. TypeScript schemas and generic calls
are compiled into specialized encoders and decoders without runtime schema lookup.
Vendored from [SnowLuma/protobuf-dsl](https://github.com/SnowLuma/protobuf-dsl)
(`protobuf-fastdsl`, commit `22e382a`), whose README declares the MIT license.
SnowLuma retains the `@snowluma/proton` package name, manifest and wire compatibility
rules described below.
## Setup
Add `@snowluma/proton: workspace:*` to the consuming package's dependencies and
enable the plugin in its Vite or Vitest configuration:
```ts
import { defineConfig } from 'vite';
import protobufVitePlugin from '@snowluma/proton/vite';
export default defineConfig({
plugins: [protobufVitePlugin()],
});
```
The plugin entry resolves to a built artifact. After editing the compiler, run
`pnpm --filter @snowluma/proton build` before testing consuming packages.
Calls without the plugin throw a diagnostic naming `@snowluma/proton/vite`.
```ts
import type { pb, pb_optional, pb_repeated, uint_32 } from '@snowluma/proton';
import { protobuf_encode, protobuf_decode } from '@snowluma/proton';
interface Profile {
id?: pb<1, uint_32>;
name?: pb<2, string>;
tags?: pb_repeated<3, string>;
state?: pb_optional<4, uint_32>;
}
const bytes = protobuf_encode<Profile>({ id: 42, name: 'alice', state: 0 });
const profile = protobuf_decode<Profile>(bytes);
```
## Schema and compiler support
`pb<N, T>` marks a singular field, `pb_repeated<N, T>` marks a repeated field,
and `pb_optional<N, T>` gives a singular field explicit wire presence.
| Types | TypeScript value | Wire format |
| --- | --- | --- |
| `uint_32`, `int_32`, `sint_32` | `number` | Varint; `sint_32` uses zigzag |
| `uint_64`, `int_64`, `sint_64` | `bigint` | Varint; `sint_64` uses zigzag |
| `bool` | `boolean` | Varint |
| `fixed_32`, `sfixed_32`, `float` | `number` | 32-bit |
| `fixed_64`, `sfixed_64` | `bigint` | 64-bit |
| `double` | `number` | 64-bit |
| `string`, `bytes` | `string`, `Uint8Array` | Length-delimited |
| Message interfaces | Object | Length-delimited |
The compiler resolves aliases, namespace imports, `export *`, relative and bare
package imports, nested generic schemas, generic functions and classes, and
forwarded callbacks. It preserves wrapper bodies and passes generated codecs
through generic calls. It also supports resolvable mapped, conditional and utility
types; ambiguous unions and unresolved schemas fail during compilation.
Only required codecs and helpers are emitted. Nested message encoders measure and
write into a shared output buffer. Returned bytes may be a view into a larger
buffer; use `byteOffset` and `byteLength`, or `bytes.slice()` when an independent
backing buffer is needed. Later encodes do not overwrite previously returned bytes.
## SnowLuma wire compatibility
- Plain `pb<>` scalars omit default values (`0`, `0n`, `false`, empty strings and
empty bytes), including fields marked with TypeScript `?`. Missing optional
singular fields decode to `null`.
- `pb_optional<>` emits present default values while omitting `null`/`undefined`.
This preserves the explicit `state=0` needed to disable whole-group mute.
- Negative `int_32` values retain SnowLuma's five-byte outbound encoding. Decoders
accept both that form and canonical ten-byte sign-extended values.
- `uint_32` values consume a complete valid varint of up to ten bytes and retain
the low 32 bits. Tags and lengths remain limited to valid 32-bit framing.
- Decoders reject truncated/overflowing fields, invalid bounds and malformed
groups. Unknown groups are skipped within their parent frame with bounded depth.
- Unknown tags retain non-enumerable metadata, available through
`protobuf_getUnknownFields` and `protobuf_getUnknownFieldMetadata`. Metadata
aggregates occurrences, retains at most 64 field/wire kinds per message, and
summarizes omitted kinds without retaining their payloads.
The previous runtime-map fallback and plugin option have been removed. Consumers
must compile encode/decode calls with the plugin.
## Migration audit
The upstream implementation was checked against Proton's source history before
replacement. The following behavior is covered by regression tests:
| Proton history | Upstream status / retained behavior |
| --- | --- |
| Initial vendor fixes (`194165f4`) and generic OIDB migration (`58028123`) | Aliased markers, nested generic fields and forwarded wrappers are covered by the new compiler and integration tests. |
| Barrel exports (`f21740a5`) and bare imports (`994633ea`) | Supported by the new source resolver; tested through bundled cross-file and package fixtures. |
| Explicit presence (`90f26ca4`) | Added `pb_optional` to the new schema pipeline and kept `?` separate from wire presence. |
| Decoder hardening (`b7d648ea`) | Upstream already checks varints and field boundaries; retained integer bounds, group handling, bounded unknown metadata and legacy outbound `int32` bytes. |
| Extended `uint32` (`b53004ee`, `613b2b1e`) | Upstream consumes full value varints; retained the existing QQ regression fixtures. |
| Cleanup (`c7428363`) | Superseded by the new compiler/source module structure. |
## Validation and layout
From the repository root:
```sh
pnpm --filter @snowluma/proton build
pnpm --filter @snowluma/proton typecheck
pnpm --filter @snowluma/proton test
```
`test/upstream` contains the imported compiler, codegen and plugin suite, adjusted
for SnowLuma's package layout and wire rules. `test/__tests__` retains Proton's
wire regression tests and adds migration compatibility cases. Consumer tests in
`packages/protocol/tests` exercise real QQ schemas with the built plugin.
```text
src/
source/ source files, TypeScript module resolution and build entries
schema/ type resolution and message construction
compiler/ call analysis, generic codec propagation and source rewriting
codegen/ wire emitters and selected helper generation
index.ts Vite plugin
package.ts package identity from package.json
runtime.ts uncompiled-call errors and unknown-field diagnostics
protobuf.d.ts public field markers and runtime declarations
```
+47
View File
@@ -0,0 +1,47 @@
{
"license": "SEE LICENSE IN LICENSE",
"name": "@snowluma/proton",
"version": "2.4.2",
"private": true,
"type": "module",
"description": "Compile-time protobuf encoder/decoder generator for SnowLuma. TypeScript interfaces → fully-inlined, zero-runtime protobuf codec functions. Vendored from SnowLuma/protobuf-dsl with edge-case fixes applied.",
"main": "./src/runtime.ts",
"types": "./protobuf.d.ts",
"exports": {
".": {
"types": "./protobuf.d.ts",
"default": "./src/runtime.ts"
},
"./vite": {
"types": "./src/index.ts",
"default": "./dist/index.js"
}
},
"scripts": {
"build": "tsup src/index.ts --format esm --out-dir dist --target node24 --external typescript --external vite",
"test": "vitest run",
"test:watch": "vitest",
"typecheck": "tsc --noEmit",
"prepare": "pnpm run build"
},
"dependencies": {
"typescript": "catalog:"
},
"peerDependencies": {
"vite": "catalog:"
},
"peerDependenciesMeta": {
"vite": {
"optional": true
}
},
"devDependencies": {
"@types/node": "catalog:",
"tsup": "^8.5.0",
"vite": "catalog:",
"vitest": "4.1.12"
},
"engines": {
"node": "^24.22.1 || >=26.12.1"
}
}
+49
View File
@@ -0,0 +1,49 @@
// ── Protobuf field markers ────────────────────────────────────────────
/** Marks a singular protobuf field: `name: pb<fieldNumber, Type>` */
export type pb<_ProtoNumber extends number, Type> = Type;
/** Marks a repeated protobuf field: `ids: pb_repeated<fieldNumber, Type>` → Type[] */
export type pb_repeated<_ProtoNumber extends number, Type> = Type[];
/**
* Marks a singular field with **explicit presence** (proto3 `optional`): it
* serialises whenever the value is present, INCLUDING its zero/default value
* — unlike `pb<>`, which omits zeros. Use when the wire peer needs the field
* present to disambiguate, e.g. an OIDB sub-command keyed on body shape where
* a zero must still appear on the wire. `name: pb_optional<fieldNumber, Type>`.
*/
export type pb_optional<_ProtoNumber extends number, Type> = Type;
// ── Protobuf primitive types ──────────────────────────────────────────
export type uint_32 = number;
export type int_32 = number;
export type uint_64 = bigint;
export type int_64 = bigint;
export type sint_32 = number;
export type sint_64 = bigint;
export type bool = boolean;
export type float = number;
export type double = number;
export type fixed_32 = number;
export type fixed_64 = bigint;
export type sfixed_32 = number;
export type sfixed_64 = bigint;
export type bytes = Uint8Array;
// ── Encode / decode (replaced at compile-time by the vite plugin) ────
export function protobuf_encode<T>(params: T): Uint8Array;
export function protobuf_decode<T>(data: Uint8Array): T;
/** Non-enumerable metadata retained for protobuf tags unknown to the schema. */
export interface ProtobufUnknownField {
fieldNumber: number;
wireType: number;
count: number;
totalByteLength: number;
}
export interface ProtobufUnknownFieldMetadata {
fields: readonly ProtobufUnknownField[];
totalOccurrences: number;
omittedOccurrences: number;
omittedByteLength: number;
}
export function protobuf_getUnknownFields(value: unknown): readonly ProtobufUnknownField[];
export function protobuf_getUnknownFieldMetadata(value: unknown): ProtobufUnknownFieldMetadata;
+13
View File
@@ -0,0 +1,13 @@
export const buffers = `let __pool;
let __poolOffset = 0;
function __allocate(size) {
if (size > 4096) return new Uint8Array(size);
let buffer = __pool;
if (!buffer || __poolOffset + size > buffer.length) {
buffer = __pool = new Uint8Array(8192);
__poolOffset = 0;
}
const start = __poolOffset;
__poolOffset += size;
return buffer.subarray(start, start + size);
}`;
+251
View File
@@ -0,0 +1,251 @@
import {
WireType,
PRIMITIVE_TYPE_MAP,
type ProtobufField,
type ProtobufMessage,
} from '../codegen/model.js';
function tagValue(field: ProtobufField, wireType = field.wireType): number {
return ((field.fieldNumber << 3) | wireType) >>> 0;
}
/** Read the low 32 bits, consuming sign-extended int32 varints without shift wraparound. */
function varintDec(
varName: string,
ind: string,
end = 'end',
maxBytes = 10,
truthy = false,
checkInitial = true,
): string {
const lastShift = (maxBytes - 1) * 7;
const lastByte = maxBytes === 5 ? 0x0f : 0x01;
return [
...(checkInitial
? [`${ind}if (offset >= ${end}) throw new Error('Truncated protobuf varint');`]
: []),
`${ind}let ${varName} = data[offset++];`,
`${ind}if (${varName} & 0x80) {`,
`${ind} let _s = 7, _b;`,
`${ind} ${varName} &= 0x7f;`,
`${ind} do {`,
`${ind} if (offset >= ${end}) throw new Error('Truncated protobuf varint');`,
`${ind} _b = data[offset++];`,
`${ind} if (_s === ${lastShift} && _b > ${lastByte}) throw new Error('Invalid protobuf varint');`,
truthy
? `${ind} ${varName} |= _b & 0x7f;`
: `${ind} if (_s < 32) ${varName} |= (_b & 0x7f) << _s;`,
`${ind} _s += 7;`,
`${ind} } while (_b & 0x80);`,
`${ind} ${varName} >>>= 0;`,
`${ind}}`,
].join('\n');
}
function varintDec64(varName: string, ind: string, end: string): string {
return [
`${ind}let _lo = 0, _hi = 0, _s = 0, _b;`,
`${ind}do {`,
`${ind} if (offset >= ${end}) throw new Error('Truncated protobuf varint');`,
`${ind} _b = data[offset++];`,
`${ind} if (_s === 63 && _b > 1) throw new Error('Invalid protobuf varint');`,
`${ind} if (_s < 28) _lo |= (_b & 0x7f) << _s;`,
`${ind} else if (_s === 28) { _lo |= (_b & 0x7f) << 28; _hi = (_b & 0x7f) >>> 4; }`,
`${ind} else _hi |= (_b & 0x7f) << (_s - 32);`,
`${ind} _s += 7;`,
`${ind}} while (_b & 0x80);`,
`${ind}const ${varName} = _hi === 0 ? BigInt(_lo >>> 0) : BigInt(_lo >>> 0) | (BigInt(_hi >>> 0) << 32n);`,
].join('\n');
}
function checkLength(length: string | number, ind: string, end: string): string {
return `${ind}if (offset + ${length} > ${end}) throw new Error('Truncated protobuf field');`;
}
export function generateDecoder(msg: ProtobufMessage): string {
const locals = msg.fields.map((field, index) => {
const keyword = field.isRepeated ? 'const' : 'let';
return ` ${keyword} _f${index} = ${getDefault(field)};`;
});
const result = msg.fields
.map(
(field, index) =>
`${field.name === '__proto__' ? '["__proto__"]' : JSON.stringify(field.name)}: _f${index}`,
)
.join(', ');
const lines = [
`function protobuf_decode_${msg.name}(data, offset = 0, end = data.length) {`,
` if (!Number.isSafeInteger(offset) || !Number.isSafeInteger(end) || offset < 0 || end > data.length || offset > end) throw new Error('Invalid protobuf bounds');`,
...locals,
` let _unknownFields = null;`,
` let _unknownFieldsByKey = null;`,
` let _unknownTotalOccurrences = 0;`,
` let _unknownOmittedOccurrences = 0;`,
` let _unknownOmittedByteLength = 0;`,
` while (offset < end) {`,
varintDec('_tag', ' ', 'end', 5, false, false),
` if ((_tag >>> 3) === 0) throw new Error('Invalid protobuf field number');`,
` switch (_tag) {`,
];
msg.fields.forEach((field, index) => {
lines.push(` case ${tagValue(field)}: {`);
lines.push(...decodeValue(field, index, ' ', 'end'));
lines.push(` break;`, ` }`);
if (field.isRepeated && !field.isMessage && field.wireType !== WireType.LengthDelim) {
lines.push(
` case ${tagValue(field, WireType.LengthDelim)}: {`,
varintDec('_len', ' ', 'end', 5),
checkLength('_len', ' ', 'end'),
` const _packedEnd = offset + _len;`,
` while (offset < _packedEnd) {`,
...decodeValue(field, index, ' ', '_packedEnd'),
` }`,
` break;`,
` }`,
);
}
});
lines.push(
` default: {`,
` const _unknownStart = offset;`,
` const wireType = _tag & 0x7;`,
` offset = __skipUnknownField(data, offset, end, wireType, _tag >>> 3);`,
` const _unknownFieldNumber = _tag >>> 3;`,
` const _unknownByteLength = offset - _unknownStart;`,
` const _unknownKey = (_unknownFieldNumber * 8) + wireType;`,
` _unknownTotalOccurrences++;`,
` const _knownUnknown = _unknownFieldsByKey?.get(_unknownKey);`,
` if (_knownUnknown) {`,
` _knownUnknown.count++;`,
` _knownUnknown.totalByteLength += _unknownByteLength;`,
` } else if ((_unknownFields?.length ?? 0) < 64) {`,
` const _unknown = {`,
` fieldNumber: _unknownFieldNumber,`,
` wireType,`,
` count: 1,`,
` totalByteLength: _unknownByteLength,`,
` };`,
` (_unknownFields ??= []).push(_unknown);`,
` (_unknownFieldsByKey ??= new Map()).set(_unknownKey, _unknown);`,
` } else {`,
` _unknownOmittedOccurrences++;`,
` _unknownOmittedByteLength += _unknownByteLength;`,
` }`,
` break;`,
` }`,
` }`,
` }`,
` const _result = { ${result} };`,
` if (_unknownTotalOccurrences > 0) Object.defineProperty(`,
` _result,`,
` Symbol.for('snowluma.proton.unknownFields'),`,
` {`,
` value: {`,
` fields: _unknownFields ?? [],`,
` totalOccurrences: _unknownTotalOccurrences,`,
` omittedOccurrences: _unknownOmittedOccurrences,`,
` omittedByteLength: _unknownOmittedByteLength,`,
` },`,
` enumerable: false,`,
` },`,
` );`,
` return _result;`,
`}`,
);
return lines.join('\n');
}
/** Shared by ordinary fields and packed repeated payloads. */
function decodeValue(field: ProtobufField, index: number, ind: string, end: string): string[] {
const { typeName, wireType, isMessage, isRepeated } = field;
const local = `_f${index}`;
const assign = (expr: string) =>
isRepeated ? `${ind}${local}.push(${expr});` : `${ind}${local} = ${expr};`;
const lines: string[] = [];
if (wireType === WireType.Bit32) {
lines.push(checkLength(4, ind, end));
}
if (wireType === WireType.Bit64) {
lines.push(checkLength(8, ind, end));
}
if (isMessage || typeName === 'string' || typeName === 'bytes') {
lines.push(varintDec('_len', ind, end, 5), checkLength('_len', ind, end));
if (isMessage) {
lines.push(assign(`protobuf_decode_${typeName}(data, offset, offset + _len)`));
} else if (typeName === 'string') {
lines.push(
assign(
`_len <= 32 ? __utf8Read(data, offset, offset + _len) : __utf8ReadLong(data, offset, offset + _len)`,
),
);
} else {
lines.push(assign(`data.slice(offset, offset + _len)`));
}
lines.push(`${ind}offset += _len;`);
} else if (typeName === 'uint_64' || typeName === 'int_64' || typeName === 'sint_64') {
lines.push(varintDec64('_val', ind, end));
const value =
typeName === 'uint_64'
? '_val'
: typeName === 'int_64'
? 'BigInt.asIntN(64, _val)'
: '__zigZagDecode64(_val)';
lines.push(assign(value));
} else if (wireType === WireType.Varint) {
lines.push(varintDec('_val', ind, end, 10, typeName === 'bool'));
const value =
typeName === 'bool'
? '_val !== 0'
: typeName === 'sint_32'
? '(_val >>> 1) ^ -(_val & 1)'
: typeName === 'int_32'
? '_val | 0'
: '_val';
lines.push(assign(value));
} else if (typeName === 'float') {
lines.push(assign('__readFloat32(data, offset)'), `${ind}offset += 4;`);
} else if (wireType === WireType.Bit32) {
const value =
'(data[offset] | (data[offset + 1] << 8) | (data[offset + 2] << 16) | (data[offset + 3] << 24))';
lines.push(
assign(typeName === 'fixed_32' ? `${value} >>> 0` : value),
`${ind}offset += 4;`,
);
} else if (typeName === 'double') {
lines.push(assign('__readFloat64(data, offset)'), `${ind}offset += 8;`);
} else if (typeName === 'fixed_64' || typeName === 'sfixed_64') {
const value =
typeName === 'fixed_64'
? '__readFixed64(data, offset)'
: 'BigInt.asIntN(64, __readFixed64(data, offset))';
lines.push(assign(value), `${ind}offset += 8;`);
} else {
throw new Error(`Unsupported protobuf field type: ${typeName}`);
}
return lines;
}
function getDefault(field: ProtobufField): string {
if (field.isRepeated) {
return '[]';
}
if (field.isOptional || field.isMessage) {
return 'null';
}
const primitive = PRIMITIVE_TYPE_MAP[field.typeName];
return primitive ? primitive.defaultValue : 'null';
}
@@ -0,0 +1,34 @@
import type { MessageRegistry } from './model.js';
export function collectDependencyClosure(
registry: MessageRegistry,
roots: Iterable<string>,
): Set<string> {
const selected = new Set<string>();
function visit(name: string): void {
if (selected.has(name)) {
return;
}
const msg = registry.get(name);
if (!msg) {
return;
}
selected.add(name);
for (const field of msg.fields) {
if (registry.has(field.typeName)) {
visit(field.typeName);
}
}
}
for (const root of roots) {
visit(root);
}
return selected;
}
+51
View File
@@ -0,0 +1,51 @@
import type { ProtobufMessage } from './model.js';
import { fieldBlock } from './fields.js';
export function generateEncoderHelpers(message: ProtobufMessage): string {
const blocks = message.fields.map((field, index) => fieldBlock(field, index, true));
return [
`function __size_${message.name}(obj, sizes) {`,
' let size = 0;',
...blocks.flatMap((block) => block.declare),
...blocks.flatMap((block) => block.size),
' return size;',
'}',
`function __write_${message.name}(obj, buf, offset, sizes) {`,
...blocks.flatMap((block) => block.declare),
...blocks.flatMap((block) => block.write),
' return offset;',
'}',
].join('\n');
}
export function generateEncoder(message: ProtobufMessage, pooled = false): string {
const allocation = pooled
? ' const buf = size <= 64 ? new Uint8Array(size) : __allocate(size);'
: ' const buf = new Uint8Array(size);';
if (message.fields.some((field) => field.isMessage)) {
return [
`function protobuf_encode_${message.name}(obj) {`,
' const sizes = [1];',
...(pooled
? [` const size = __size_${message.name}(obj, sizes);`, allocation]
: [` const buf = new Uint8Array(__size_${message.name}(obj, sizes));`]),
` __write_${message.name}(obj, buf, 0, sizes);`,
' return buf;',
'}',
].join('\n');
}
const blocks = message.fields.map((field, index) => fieldBlock(field, index, false));
return [
`function protobuf_encode_${message.name}(obj) {`,
' let size = 0;',
...blocks.flatMap((block) => block.declare),
...blocks.flatMap((block) => block.size),
allocation,
' let offset = 0;',
...blocks.flatMap((block) => block.write),
' return buf;',
'}',
].join('\n');
}
+141
View File
@@ -0,0 +1,141 @@
import { WireType, type ProtobufField } from './model.js';
import { computeTagBytes, varintSize, writeTag, writeVarint } from './write.js';
export interface FieldBlock {
declare: string[];
size: string[];
write: string[];
}
export function fieldBlock(field: ProtobufField, index: number, shared: boolean): FieldBlock {
const { fieldNumber, wireType, typeName, isRepeated, isMessage } = field;
const variable = `_f${index}`;
const item = isRepeated ? `${variable}[_i]` : variable;
const cached = `_c${index}`;
const slot = isRepeated ? `${cached}[_i]` : cached;
const indent = isRepeated ? ' ' : ' ';
const tag = computeTagBytes(fieldNumber, wireType).length;
const condition = isRepeated
? `${variable} != null && ${variable}.length > 0`
: presence(field, variable);
const start = [` if (${condition}) {`];
const loop = isRepeated ? [` for (let _i = 0; _i < ${variable}.length; _i++) {`] : [];
const end = [...(isRepeated ? [' }'] : []), ' }'];
const declare = [` const ${variable} = obj[${JSON.stringify(field.name)}];`];
const setup: string[] = [];
const size: string[] = [];
const write: string[] = [writeTag(fieldNumber, wireType, indent)];
const emit = (lines: string[]) => lines.map((line) => indent + line);
if (wireType === WireType.LengthDelim) {
if (isMessage) {
size.push(
...emit([
'const _slot = sizes.length;',
'sizes.push(0);',
`const _len = __size_${typeName}(${item}, sizes);`,
'sizes[_slot] = _len;',
]),
);
write.push(...emit(['const _len = sizes[sizes[0]++];']));
} else if (typeName === 'string') {
if (shared) {
size.push(...emit([`const _len = __utf8Len(${item});`, 'sizes.push(_len);']));
write.push(...emit(['const _len = sizes[sizes[0]++];']));
} else {
declare.push(` let ${cached};`);
if (isRepeated) {
setup.push(` ${cached} = new Array(${variable}.length);`);
}
size.push(...emit([`const _len = __utf8Len(${item});`, `${slot} = _len;`]));
write.push(...emit([`const _len = ${slot};`]));
}
} else {
size.push(...emit([`const _len = ${item}.length;`]));
write.push(...emit([`const _len = ${item}.length;`]));
}
size.push(...emit([`size += ${tag} + _len;`]), varintSize('_len', indent));
write.push(
writeVarint('_len', indent),
...emit(
isMessage
? [`offset = __write_${typeName}(${item}, buf, offset, sizes);`]
: typeName === 'string'
? [
`offset = _len >= 128 ? __utf8WriteLong(buf, offset, ${item}) : _len === ${item}.length ? __asciiWrite(buf, offset, ${item}) : __utf8Write(buf, offset, ${item});`,
]
: [`buf.set(${item}, offset);`, 'offset += _len;'],
),
);
} else if (typeName === 'bool') {
size.push(...emit([`size += ${tag + 1};`]));
write.push(...emit([`buf[offset++] = ${item} ? 1 : 0;`]));
} else if (wireType === WireType.Varint) {
if (typeName.endsWith('_64')) {
const normalized =
typeName === 'sint_64'
? `__zigZagEncode64(${item})`
: `BigInt.asUintN(64, ${item})`;
size.push(...emit([`size += ${tag} + __varint64Size(${normalized});`]));
write.push(...emit([`offset = __writeVarint64(buf, offset, ${normalized});`]));
} else {
// Preserve QQ's established five-byte negative int32 encoding. Decoders
// also accept the canonical sign-extended ten-byte representation.
const normalized =
typeName === 'sint_32'
? `((${item} << 1) ^ (${item} >> 31)) >>> 0`
: `${item} >>> 0`;
size.push(
...emit([`const _val = ${normalized};`, `size += ${tag};`]),
varintSize('_val', indent),
);
write.push(writeVarint(normalized, indent));
}
} else {
size.push(...emit([`size += ${tag + (wireType === WireType.Bit32 ? 4 : 8)};`]));
if (typeName === 'float' || typeName === 'double') {
write.push(
...emit([
`offset = __writeFloat${typeName === 'float' ? 32 : 64}(buf, offset, ${item});`,
]),
);
} else if (wireType === WireType.Bit64) {
write.push(...emit([`offset = __writeFixed64(buf, offset, ${item});`]));
} else {
write.push(
...emit([
`const _val = ${item};`,
...[0, 8, 16, 24].map((shift) => `buf[offset++] = (_val >>> ${shift}) & 0xff;`),
]),
);
}
}
return {
declare,
size: [...start, ...setup, ...loop, ...size, ...end],
write: [...start, ...loop, ...write, ...end],
};
}
function presence(field: ProtobufField, value: string): string {
// TypeScript optionality controls missing decoded values, not wire presence.
if (field.explicitPresence || field.isMessage) {
return `${value} != null`;
}
if (field.typeName === 'bool') {
return `${value} === true`;
}
if (field.typeName === 'bytes') {
return `${value} != null && ${value}.length > 0`;
}
const empty = field.typeName === 'string' ? '""' : field.typeName.endsWith('_64') ? '0n' : '0';
return `${value} != null && ${value} !== ${empty}`;
}
+99
View File
@@ -0,0 +1,99 @@
import { WireType, type MessageRegistry, type ProtobufMessage } from './model.js';
import { generateEncoder, generateEncoderHelpers } from './encoder.js';
import { generateDecoder } from './decoder.js';
import { collectDependencyClosure } from '../codegen/dependencies.js';
import { helpers } from './helpers.js';
/**
* Generate fully self-contained encode/decode source code.
* No runtime imports needed — all wire-format logic is inlined.
*/
export interface CodegenOptions {
encodeRoots?: Iterable<string>;
decodeRoots?: Iterable<string>;
}
export function generateCode(registry: MessageRegistry, options: CodegenOptions = {}): string {
if (registry.size === 0) {
return '';
}
const encodeRoots = new Set(options.encodeRoots ?? registry.keys());
const decodeNames = collectDependencyClosure(registry, options.decodeRoots ?? registry.keys());
const nestedRoots = [...encodeRoots].filter((name) =>
registry.get(name)?.fields.some((field) => field.isMessage),
);
const encoderHelpers = collectDependencyClosure(registry, nestedRoots);
const bounds = new Map<string, number>();
function sizeBound(message: ProtobufMessage): number {
const cached = bounds.get(message.name);
if (cached !== undefined) {
return cached;
}
bounds.set(message.name, Infinity);
let size = 0;
for (const field of message.fields) {
if (field.isRepeated || (!field.isMessage && field.wireType === WireType.LengthDelim)) {
return Infinity;
}
// A scalar needs at most 5 tag bytes and 10 payload bytes.
size += 16;
if (field.isMessage) {
size += sizeBound(registry.get(field.typeName)!);
}
if (size > 64) {
return Infinity;
}
}
bounds.set(message.name, size);
return size;
}
const parts: string[] = [];
for (const msg of registry.values()) {
if (encodeRoots.has(msg.name)) {
parts.push(generateEncoder(msg, sizeBound(msg) > 64));
}
if (encoderHelpers.has(msg.name)) {
parts.push(generateEncoderHelpers(msg));
}
if (decodeNames.has(msg.name)) {
parts.push(generateDecoder(msg));
}
}
const body = parts.join('\n');
const selected = new Set<string>();
function include(text: string): void {
for (const match of text.matchAll(/\b__\w+/g)) {
const name = match[0];
if (selected.has(name) || !helpers.has(name)) {
continue;
}
selected.add(name);
include(helpers.get(name)!);
}
}
include(body);
return [...helpers]
.filter(([name]) => selected.has(name))
.map(([, text]) => text)
.concat(body)
.join('\n');
}
+98
View File
@@ -0,0 +1,98 @@
import { utf8 } from './utf8.js';
import { unknownFields } from './unknown.js';
import { buffers } from './buffer.js';
const CODEGEN_PREAMBLE = `${utf8}
${buffers}
${unknownFields}
const __scratch = new DataView(new ArrayBuffer(8));
function __varint64Size(value) {
if (value < 0x80n) return 1;
const hi = Number(value >> 32n);
const bits = hi ? 64 - Math.clz32(hi) : 32 - Math.clz32(Number(value));
return ((bits + 6) / 7) | 0;
}
function __writeVarint64(buf, offset, value) {
let lo = Number(value & 0xffffffffn) >>> 0;
let hi = Number(value >> 32n) >>> 0;
while (hi || lo > 0x7f) {
buf[offset++] = (lo & 0x7f) | 0x80;
lo = ((lo >>> 7) | (hi << 25)) >>> 0;
hi >>>= 7;
}
buf[offset++] = lo;
return offset;
}
function __zigZagEncode64(value) {
value = BigInt.asIntN(64, value);
return BigInt.asUintN(64, (value << 1n) ^ (value >> 63n));
}
function __zigZagDecode64(value) {
return BigInt.asIntN(64, (value >> 1n) ^ -(value & 1n));
}
function __writeFloat32(buf, offset, value) {
__scratch.setFloat32(0, value, true);
buf[offset++] = __scratch.getUint8(0);
buf[offset++] = __scratch.getUint8(1);
buf[offset++] = __scratch.getUint8(2);
buf[offset++] = __scratch.getUint8(3);
return offset;
}
function __readFloat32(data, offset) {
__scratch.setUint8(0, data[offset]);
__scratch.setUint8(1, data[offset + 1]);
__scratch.setUint8(2, data[offset + 2]);
__scratch.setUint8(3, data[offset + 3]);
return __scratch.getFloat32(0, true);
}
function __writeFloat64(buf, offset, value) {
__scratch.setFloat64(0, value, true);
buf[offset++] = __scratch.getUint8(0);
buf[offset++] = __scratch.getUint8(1);
buf[offset++] = __scratch.getUint8(2);
buf[offset++] = __scratch.getUint8(3);
buf[offset++] = __scratch.getUint8(4);
buf[offset++] = __scratch.getUint8(5);
buf[offset++] = __scratch.getUint8(6);
buf[offset++] = __scratch.getUint8(7);
return offset;
}
function __readFloat64(data, offset) {
__scratch.setUint8(0, data[offset]);
__scratch.setUint8(1, data[offset + 1]);
__scratch.setUint8(2, data[offset + 2]);
__scratch.setUint8(3, data[offset + 3]);
__scratch.setUint8(4, data[offset + 4]);
__scratch.setUint8(5, data[offset + 5]);
__scratch.setUint8(6, data[offset + 6]);
__scratch.setUint8(7, data[offset + 7]);
return __scratch.getFloat64(0, true);
}
function __writeFixed64(buf, offset, value) {
value = BigInt.asUintN(64, value);
const lo = Number(value & 0xffffffffn), hi = Number(value >> 32n);
buf[offset++] = lo;
buf[offset++] = lo >>> 8;
buf[offset++] = lo >>> 16;
buf[offset++] = lo >>> 24;
buf[offset++] = hi;
buf[offset++] = hi >>> 8;
buf[offset++] = hi >>> 16;
buf[offset++] = hi >>> 24;
return offset;
}
function __readFixed64(data, offset) {
const lo = (data[offset] | (data[offset + 1] << 8) | (data[offset + 2] << 16) | (data[offset + 3] << 24)) >>> 0;
const hi = (data[offset + 4] | (data[offset + 5] << 8) | (data[offset + 6] << 16) | (data[offset + 7] << 24)) >>> 0;
return BigInt(lo) | (BigInt(hi) << 32n);
}`;
export const helpers = new Map<string, string>();
const helperStarts = [...CODEGEN_PREAMBLE.matchAll(/^(?:function|const|let) (__\w+)/gm)];
for (let i = 0; i < helperStarts.length; i++) {
helpers.set(
helperStarts[i][1],
CODEGEN_PREAMBLE.slice(helperStarts[i].index, helperStarts[i + 1]?.index).trim(),
);
}
+42
View File
@@ -0,0 +1,42 @@
export enum WireType {
Varint = 0,
Bit64 = 1,
LengthDelim = 2,
Bit32 = 5,
}
export interface ProtobufField {
name: string;
fieldNumber: number;
typeName: string;
wireType: WireType;
isMessage: boolean;
isOptional: boolean;
isRepeated: boolean;
explicitPresence?: boolean;
}
export interface ProtobufMessage {
name: string;
fields: ProtobufField[];
}
export type MessageRegistry = Map<string, ProtobufMessage>;
export const PRIMITIVE_TYPE_MAP: Record<string, { wireType: WireType; defaultValue: string }> = {
uint_32: { wireType: WireType.Varint, defaultValue: '0' },
int_32: { wireType: WireType.Varint, defaultValue: '0' },
uint_64: { wireType: WireType.Varint, defaultValue: '0n' },
int_64: { wireType: WireType.Varint, defaultValue: '0n' },
sint_32: { wireType: WireType.Varint, defaultValue: '0' },
sint_64: { wireType: WireType.Varint, defaultValue: '0n' },
bool: { wireType: WireType.Varint, defaultValue: 'false' },
string: { wireType: WireType.LengthDelim, defaultValue: '""' },
bytes: { wireType: WireType.LengthDelim, defaultValue: 'new Uint8Array(0)' },
float: { wireType: WireType.Bit32, defaultValue: '0' },
double: { wireType: WireType.Bit64, defaultValue: '0' },
fixed_32: { wireType: WireType.Bit32, defaultValue: '0' },
fixed_64: { wireType: WireType.Bit64, defaultValue: '0n' },
sfixed_32: { wireType: WireType.Bit32, defaultValue: '0' },
sfixed_64: { wireType: WireType.Bit64, defaultValue: '0n' },
};
+60
View File
@@ -0,0 +1,60 @@
// Bound unknown fields and groups to the current message frame.
export const unknownFields = `
function __readVarint32(data, offset, end) {
let value = 0;
for (let i = 0; i < 5; i++) {
if (offset >= end) throw new Error('protobuf truncated uint32 varint');
const byte = data[offset++];
if (i === 4 && (byte & 0xf0) !== 0) throw new Error('protobuf uint32 varint overflow');
value += (byte & 0x7f) * (2 ** (i * 7));
if ((byte & 0x80) === 0) return [value >>> 0, offset];
}
throw new Error('protobuf uint32 varint overflow');
}
function __skipVarint(data, offset, end) {
for (let i = 0; i < 10; i++) {
if (offset >= end) throw new Error('protobuf truncated varint');
const byte = data[offset++];
if (i === 9 && byte > 1) throw new Error('protobuf varint overflow');
if ((byte & 0x80) === 0) return offset;
}
throw new Error('protobuf varint overflow');
}
function __checkedEnd(offset, length, end) {
if (length > end - offset) throw new Error('protobuf length-delimited field exceeds parent bounds');
return offset + length;
}
function __skipUnknownField(data, offset, end, wireType, fieldNumber, depth = 0) {
if (depth > 64) throw new Error('protobuf group nesting exceeds 64');
if (wireType === 0) return __skipVarint(data, offset, end);
if (wireType === 1) {
if (end - offset < 8) throw new Error('protobuf truncated fixed64 field');
return offset + 8;
}
if (wireType === 2) {
const [length, next] = __readVarint32(data, offset, end);
return __checkedEnd(next, length, end);
}
if (wireType === 3) {
while (offset < end) {
const [tag, next] = __readVarint32(data, offset, end);
offset = next;
const nestedField = tag >>> 3;
const nestedWire = tag & 0x7;
if (nestedField === 0) throw new Error('protobuf field number 0 is invalid');
if (nestedWire === 4) {
if (nestedField !== fieldNumber) throw new Error('protobuf mismatched end-group tag');
return offset;
}
offset = __skipUnknownField(data, offset, end, nestedWire, nestedField, depth + 1);
}
throw new Error('protobuf unterminated group');
}
if (wireType === 4) throw new Error('protobuf unexpected end-group tag');
if (wireType === 5) {
if (end - offset < 4) throw new Error('protobuf truncated fixed32 field');
return offset + 4;
}
throw new Error('protobuf invalid wire type ' + wireType);
}
`;
+159
View File
@@ -0,0 +1,159 @@
export const utf8 = `const __td = new TextDecoder();
const __te = new TextEncoder();
const __nonAscii = /[^\\x00-\\x7f]/;
let __utf8Scratch;
const __tdStrict = new TextDecoder('utf-8', { fatal: true });
function __utf8LenLong(value) {
const buffer = __utf8Scratch ??= new Uint8Array(4096);
let length = 0;
for (;;) {
const result = __te.encodeInto(value, buffer);
length += result.written;
if (result.read === value.length) return length;
value = value.slice(result.read);
}
}
function __utf8Len(value) {
if (value.length >= 32) return __utf8LenLong(value);
if (value.length >= 8 && value.charCodeAt(0) < 0x80 && !__nonAscii.test(value)) return value.length;
let length = 0;
for (let i = 0; i < value.length; i++) {
const code = value.charCodeAt(i);
if (code < 0x80) {
length++;
continue;
}
if (code < 0x800) {
length += 2;
continue;
}
if ((code & 0xfc00) === 0xd800) {
if (i + 1 < value.length) {
const next = value.charCodeAt(i + 1);
if ((next & 0xfc00) === 0xdc00) {
length += 4;
i++;
continue;
}
}
length += 3;
continue;
}
if ((code & 0xfc00) === 0xdc00) {
length += 3;
continue;
}
length += 3;
}
return length;
}
function __utf8Write(buf, offset, value) {
for (let i = 0; i < value.length; i++) {
let code = value.charCodeAt(i);
if (code < 0x80) {
buf[offset++] = code;
continue;
}
if (code < 0x800) {
buf[offset++] = 0xc0 | (code >> 6);
buf[offset++] = 0x80 | (code & 0x3f);
continue;
}
if ((code & 0xfc00) === 0xd800) {
if (i + 1 < value.length) {
const next = value.charCodeAt(i + 1);
if ((next & 0xfc00) === 0xdc00) {
const point = ((code - 0xd800) << 10) + (next - 0xdc00) + 0x10000;
buf[offset++] = 0xf0 | (point >> 18);
buf[offset++] = 0x80 | ((point >> 12) & 0x3f);
buf[offset++] = 0x80 | ((point >> 6) & 0x3f);
buf[offset++] = 0x80 | (point & 0x3f);
i++;
continue;
}
}
code = 0xfffd;
} else if ((code & 0xfc00) === 0xdc00) {
code = 0xfffd;
}
buf[offset++] = 0xe0 | (code >> 12);
buf[offset++] = 0x80 | ((code >> 6) & 0x3f);
buf[offset++] = 0x80 | (code & 0x3f);
}
return offset;
}
function __utf8WriteLong(buf, offset, value) {
return offset + __te.encodeInto(value, buf.subarray(offset)).written;
}
function __asciiWrite(buf, offset, value) {
let i = 0;
for (; i + 7 < value.length; i += 8) {
buf[offset++] = value.charCodeAt(i);
buf[offset++] = value.charCodeAt(i + 1);
buf[offset++] = value.charCodeAt(i + 2);
buf[offset++] = value.charCodeAt(i + 3);
buf[offset++] = value.charCodeAt(i + 4);
buf[offset++] = value.charCodeAt(i + 5);
buf[offset++] = value.charCodeAt(i + 6);
buf[offset++] = value.charCodeAt(i + 7);
}
for (; i < value.length; i++) buf[offset++] = value.charCodeAt(i);
return offset;
}
function __utf8ReadLong(data, start, end) {
const bytes = data.subarray(start, end);
if (end - start < 128) return __td.decode(bytes);
try {
return __tdStrict.decode(bytes);
} catch {
return __td.decode(bytes);
}
}
function __utf8Read(data, start, end) {
let value = '', offset = start;
while (offset + 7 < end) {
const a = data[offset], b = data[offset + 1], c = data[offset + 2], d = data[offset + 3];
const e = data[offset + 4], f = data[offset + 5], g = data[offset + 6], h = data[offset + 7];
if ((a | b | c | d | e | f | g | h) & 0x80) break;
value += String.fromCharCode(a, b, c, d, e, f, g, h);
offset += 8;
}
if (offset + 3 < end) {
const a = data[offset], b = data[offset + 1], c = data[offset + 2], d = data[offset + 3];
if (!((a | b | c | d) & 0x80)) {
value += String.fromCharCode(a, b, c, d);
offset += 4;
}
}
while (offset < end) {
const a = data[offset++];
if (a < 0x80) {
value += String.fromCharCode(a);
} else if (a >= 0xc2 && a < 0xe0 && offset < end) {
const b = data[offset++];
if ((b & 0xc0) !== 0x80) return __td.decode(data.subarray(start, end));
value += String.fromCharCode(((a & 0x1f) << 6) | (b & 0x3f));
} else if (a >= 0xe0 && a < 0xf0 && offset + 1 < end) {
const b = data[offset++], c = data[offset++];
if ((b & 0xc0) !== 0x80 || (c & 0xc0) !== 0x80 ||
(a === 0xe0 && b < 0xa0) || (a === 0xed && b >= 0xa0)) {
return __td.decode(data.subarray(start, end));
}
const code = ((a & 0x0f) << 12) | ((b & 0x3f) << 6) | (c & 0x3f);
// Match TextDecoder's handling of a leading BOM.
if (code !== 0xfeff || offset !== start + 3) value += String.fromCharCode(code);
} else if (a >= 0xf0 && a < 0xf5 && offset + 2 < end) {
const b = data[offset++], c = data[offset++], d = data[offset++];
if ((b & 0xc0) !== 0x80 || (c & 0xc0) !== 0x80 || (d & 0xc0) !== 0x80 ||
(a === 0xf0 && b < 0x90) || (a === 0xf4 && b >= 0x90)) {
return __td.decode(data.subarray(start, end));
}
const code = (((a & 7) << 18) | ((b & 0x3f) << 12) | ((c & 0x3f) << 6) | (d & 0x3f)) - 0x10000;
value += String.fromCharCode(0xd800 | (code >>> 10), 0xdc00 | (code & 0x3ff));
} else {
return __td.decode(data.subarray(start, end));
}
}
return value;
}
`;
+56
View File
@@ -0,0 +1,56 @@
export function computeTagBytes(fieldNumber: number, wireType: number): number[] {
let value = ((fieldNumber << 3) | wireType) >>> 0;
const bytes: number[] = [];
while (value > 0x7f) {
bytes.push((value & 0x7f) | 0x80);
value >>>= 7;
}
bytes.push(value & 0x7f);
return bytes;
}
export function writeTag(fieldNumber: number, wireType: number, ind: string): string {
return computeTagBytes(fieldNumber, wireType)
.map((byte) => `${ind}buf[offset++] = ${byte};`)
.join('\n');
}
export function varintSize(varName: string, ind: string, signed = false): string {
const negativeSize = signed ? `${varName} < 0 ? 10 : ` : '';
return `${ind}size += ${negativeSize}${varName} < 0x80 ? 1 : ${varName} < 0x4000 ? 2 : ${varName} < 0x200000 ? 3 : ${varName} < 0x10000000 ? 4 : 5;`;
}
export function writeVarint(expr: string, ind: string, signed = false): string {
return [
`${ind}let _v = ${expr};`,
...(signed
? [
`${ind}if (_v < 0) {`,
`${ind} offset = __writeVarint64(buf, offset, BigInt.asUintN(64, BigInt(_v)));`,
]
: []),
`${ind}${signed ? '} else if' : 'if'} (_v < 0x80) {`,
`${ind} buf[offset++] = _v;`,
`${ind}} else if (_v < 0x4000) {`,
`${ind} buf[offset++] = (_v & 0x7f) | 0x80;`,
`${ind} buf[offset++] = _v >>> 7;`,
`${ind}} else if (_v < 0x200000) {`,
`${ind} buf[offset++] = (_v & 0x7f) | 0x80;`,
`${ind} buf[offset++] = ((_v >>> 7) & 0x7f) | 0x80;`,
`${ind} buf[offset++] = _v >>> 14;`,
`${ind}} else if (_v < 0x10000000) {`,
`${ind} buf[offset++] = (_v & 0x7f) | 0x80;`,
`${ind} buf[offset++] = ((_v >>> 7) & 0x7f) | 0x80;`,
`${ind} buf[offset++] = ((_v >>> 14) & 0x7f) | 0x80;`,
`${ind} buf[offset++] = _v >>> 21;`,
`${ind}} else {`,
`${ind} buf[offset++] = (_v & 0x7f) | 0x80;`,
`${ind} buf[offset++] = ((_v >>> 7) & 0x7f) | 0x80;`,
`${ind} buf[offset++] = ((_v >>> 14) & 0x7f) | 0x80;`,
`${ind} buf[offset++] = ((_v >>> 21) & 0x7f) | 0x80;`,
`${ind} buf[offset++] = _v >>> 28;`,
`${ind}}`,
].join('\n');
}
+559
View File
@@ -0,0 +1,559 @@
import { unique } from './edits.js';
import { Callbacks, mentionsGeneric } from './callbacks.js';
import ts from 'typescript';
import { packageName } from '../package.js';
import { SourceContext, fail, runtimeImportName } from '../source/context.js';
import { SchemaResolver } from '../schema/resolver.js';
import type { TypeEnvironment, TypeValue } from '../schema/types.js';
import {
callable,
constructorOf,
type Owner,
type Plan,
type Requirement,
type Site,
} from './model.js';
import { CallTargets } from './targets.js';
export class CallAnalysis {
readonly plans = new Map<Owner, Plan>();
readonly sites = new Map<ts.SourceFile, Site[]>();
readonly targets: CallTargets;
readonly callbacks = new Callbacks(this);
private readonly runtimePlans = new Map<string, Plan>();
constructor(
readonly context: SourceContext,
readonly resolver: SchemaResolver,
) {
this.targets = new CallTargets(context, resolver, this);
}
runtimePlan(fn: Requirement['fn']): Plan {
let plan = this.runtimePlans.get(fn);
if (!plan) {
const source = ts.createSourceFile(
`${fn}.internal.ts`,
`function ${fn}<T>(value: T) {}`,
ts.ScriptTarget.Latest,
true,
);
plan = this.plan(source.statements[0] as ts.FunctionDeclaration);
plan.runtime = fn;
const type = plan.env.get('T')!;
const requirement = { key: `${fn}:${type.key}`, fn, type, node: plan.node };
plan.requirements.set(requirement.key, requirement);
this.runtimePlans.set(fn, plan);
}
return plan;
}
scan(source: ts.SourceFile): void {
if (this.sites.has(source)) {
return;
}
const sites: Site[] = [];
this.sites.set(source, sites);
if (source.isDeclarationFile) {
return;
}
for (const statement of source.statements) {
if (
(!ts.isImportDeclaration(statement) && !ts.isExportDeclaration(statement)) ||
!statement.moduleSpecifier ||
!ts.isStringLiteral(statement.moduleSpecifier)
) {
continue;
}
const imported = this.context.module(statement.moduleSpecifier.text, source);
if (
imported &&
!imported.isDeclarationFile &&
!imported.fileName.includes('/node_modules/')
) {
this.scan(imported);
}
}
let env: TypeEnvironment = new Map();
let owner: Plan | undefined;
const visit = (node: ts.Node): void => {
this.callbacks.record(node);
if (
ts.isTypeNode(node) ||
ts.isInterfaceDeclaration(node) ||
ts.isImportDeclaration(node) ||
ts.isExportDeclaration(node)
) {
return;
}
const outerEnv = env;
const outerOwner = owner;
if (ts.isClassDeclaration(node)) {
this.plan(node);
}
if ((callable(node) || ts.isClassDeclaration(node)) && node.typeParameters?.length) {
owner = this.plan(node);
env = owner.env;
}
if (ts.isCallExpression(node) || ts.isNewExpression(node)) {
let fn = ts.isCallExpression(node)
? this.protobufFunction(node.expression, source)
: undefined;
const target = fn ? undefined : this.targets.target(node.expression, source, env);
if (!fn && ts.isCallExpression(node) && target) {
fn = this.plan(target.node).runtime;
}
if (fn) {
const typeNode =
node.typeArguments?.[0] ??
(fn === 'protobuf_encode' && node.arguments?.[0]
? this.targets.infer(node.arguments[0], source)
: undefined);
if (!typeNode) {
fail(
node,
'Cannot infer protobuf schema; supply an explicit type argument.',
);
}
const type = this.resolver.type(typeNode, typeNode.getSourceFile(), env);
const direct: Requirement = { key: `${fn}:${type.key}`, fn, type, node };
if (type.dynamic && owner) {
owner.requirements.set(direct.key, direct);
}
sites.push({ node, owner, direct });
} else {
if (target) {
const plan = this.plan(target.node);
sites.push({
node,
owner,
target: plan,
bindings: target.bindings,
typeArgs: node.typeArguments,
env,
});
this.scan(target.node.getSourceFile());
}
}
}
ts.forEachChild(node, visit);
env = outerEnv;
owner = outerOwner;
};
visit(source);
}
propagate(): void {
const consumers = new Map<
Plan,
{ owner?: Plan; bindings: () => Map<string, TypeValue> }[]
>();
const connect = (
target: Plan,
owner: Plan | undefined,
bindings: () => Map<string, TypeValue>,
) => {
const list = consumers.get(target) ?? [];
list.push({ owner, bindings });
consumers.set(target, list);
};
for (const sites of this.sites.values()) {
for (const site of sites) {
if (site.target) {
connect(site.target, site.owner, () => this.bindings(site));
}
}
}
for (const adapter of this.callbacks.connections()) {
connect(adapter.target, adapter.owner, adapter.bindings);
}
for (const plan of this.plans.values()) {
if (plan.base) {
connect(plan.base.plan, plan, () => plan.base!.bindings);
}
}
const queue = [...this.plans.values()].flatMap((plan) =>
[...plan.requirements.values()].map((requirement) => ({ plan, requirement })),
);
for (let index = 0; index < queue.length; index++) {
const { plan, requirement } = queue[index];
for (const consumer of consumers.get(plan) ?? []) {
const instantiated = this.instantiate(requirement, consumer.bindings());
const owner = consumer.owner;
if (
!instantiated.type.dynamic ||
!owner ||
owner.requirements.has(instantiated.key)
) {
continue;
}
if (owner.requirements.size >= 128) {
fail(owner.node, 'Generic codec expansion does not converge.');
}
owner.requirements.set(instantiated.key, instantiated);
queue.push({ plan: owner, requirement: instantiated });
}
}
}
instantiate(req: Requirement, bindings: Map<string, TypeValue>): Requirement {
const type = this.resolver.substitute(req.type, bindings);
return { ...req, type, key: `${req.fn}:${type.key}` };
}
bindings(site: Site, partial = false): Map<string, TypeValue> {
if (!site.bound) {
const plan = site.target!;
// Ordinary calls cannot contribute generic codec bindings. Resolving
// their parameter types would treat unrelated application data as schemas.
if (plan.env.size === 0 && !plan.base) {
site.bound = true;
return site.bindings!;
}
if (!site.typeArgs?.length) {
this.parameters(plan.node)?.forEach((param, index) => {
const argument = site.node.arguments?.[index];
const actual =
argument && this.targets.infer(argument, site.node.getSourceFile());
if (
param.type &&
actual &&
mentionsGeneric(param.type, this.environment(param)) &&
!this.targets.signature(param.type, plan.env)
) {
// Matching generic containers expose their arguments directly;
// unrelated members need not be expanded as protobuf schemas.
const pattern = param.type;
if (
ts.isTypeReferenceNode(pattern) &&
ts.isTypeReferenceNode(actual) &&
pattern.typeArguments?.length &&
pattern.typeArguments.length === actual.typeArguments?.length &&
this.context.typeName(pattern.typeName, pattern.getSourceFile())
?.node &&
this.context.typeName(pattern.typeName, pattern.getSourceFile())
?.node ===
this.context.typeName(actual.typeName, actual.getSourceFile())?.node
) {
pattern.typeArguments.forEach((type, i) =>
this.unify(
this.resolver.type(
type,
type.getSourceFile(),
this.environment(param),
),
partial
? this.resolver.lazy(
actual.typeArguments![i],
actual.getSourceFile(),
site.env!,
)
: this.resolver.type(
actual.typeArguments![i],
actual.getSourceFile(),
site.env,
),
site.bindings!,
site.node,
),
);
return;
}
this.unify(
this.resolver.type(
param.type,
param.getSourceFile(),
this.environment(param),
),
this.resolver.type(actual, actual.getSourceFile(), site.env),
site.bindings!,
site.node,
);
}
});
const bases: NonNullable<Plan['base']>[] = [];
for (let base = plan.base; base; base = base.plan.base) {
bases.push(base);
}
for (const base of bases.reverse()) {
for (const [key, pattern] of base.bindings) {
const actual = site.bindings!.get(key);
if (actual) {
this.unify(pattern, actual, site.bindings!, site.node);
}
}
}
}
const local = this.targets.environment({ node: plan.node, bindings: site.bindings! });
for (let i = 0; i < (plan.node.typeParameters?.length ?? 0); i++) {
const param = plan.node.typeParameters![i];
const key = plan.env.get(param.name.text)!.key;
const arg = site.typeArgs?.[i];
const actual =
site.bindings!.get(key) ??
(arg
? partial
? this.resolver.lazy(arg, arg.getSourceFile(), site.env!)
: this.resolver.type(arg, arg.getSourceFile(), site.env!)
: param.default
? this.resolver.type(param.default, param.getSourceFile(), local)
: undefined);
if (!actual && partial) {
continue;
}
if (!actual) {
fail(
site.node,
`Cannot infer ${param.name.text}; supply an explicit generic argument.`,
);
}
site.bindings!.set(key, actual);
local.set(param.name.text, actual);
}
site.bound = !partial;
}
return site.bindings!;
}
unify(
pattern: TypeValue,
actual: TypeValue,
bindings: Map<string, TypeValue>,
node: ts.Node,
seen = new Set<string>(),
): void {
const key = `${pattern.key}:${actual.key}`;
if (seen.has(key)) {
return;
}
seen.add(key);
if (pattern.kind === 'parameter') {
const previous = bindings.get(pattern.key);
if (previous && previous.key !== actual.key) {
fail(
node,
`Conflicting inference for ${pattern.name}; supply an explicit type argument.`,
);
}
bindings.set(pattern.key, actual);
} else if (
pattern.application &&
actual.application &&
pattern.application.declaration === actual.application.declaration
) {
pattern.application.arguments.forEach((type, index) =>
this.unify(type, actual.application!.arguments[index], bindings, node, seen),
);
} else if (pattern.fields && actual.fields) {
for (const [name, property] of pattern.fields) {
const field = actual.fields.get(name);
if (field) {
this.unify(property.type, field.type, bindings, node, seen);
}
}
} else if (
pattern.values &&
actual.values &&
pattern.values.length === actual.values.length
) {
pattern.values.forEach((type, index) =>
this.unify(type, actual.values![index], bindings, node, seen),
);
}
}
owner(node: ts.Node): Owner | undefined {
for (let parent = node.parent; parent; parent = parent.parent) {
if (
(callable(parent) || ts.isClassDeclaration(parent)) &&
parent.typeParameters?.length
) {
return parent;
}
}
return;
}
environment(node: ts.Node): TypeEnvironment {
const owners: Owner[] = [];
for (let parent: ts.Node | undefined = node; parent; parent = parent.parent) {
if (
(callable(parent) || ts.isClassDeclaration(parent)) &&
parent.typeParameters?.length
) {
owners.push(parent);
}
}
const env: TypeEnvironment = new Map();
for (const owner of owners.reverse()) {
for (const param of owner.typeParameters!) {
env.set(param.name.text, this.resolver.parameter(param.name.text, owner));
}
}
return env;
}
plan(node: Owner): Plan {
let plan = this.plans.get(node);
if (!plan) {
plan = { node, env: this.environment(node), requirements: new Map(), names: new Map() };
this.plans.set(node, plan);
if (ts.isClassDeclaration(node)) {
const heritage = node.heritageClauses?.find(
(clause) => clause.token === ts.SyntaxKind.ExtendsKeyword,
)?.types[0];
if (heritage) {
const target = this.targets.target(
heritage.expression,
node.getSourceFile(),
plan.env,
);
if (target && ts.isClassDeclaration(target.node)) {
const base = this.plan(target.node);
for (
let ancestor: Plan | undefined = base;
ancestor;
ancestor = ancestor.base?.plan
) {
if (ancestor === plan) {
fail(heritage, 'Circular class inheritance.');
}
}
const args = this.resolver.arguments(
target.node.typeParameters,
heritage.typeArguments,
node.getSourceFile(),
plan.env,
target.node,
);
const bindings = new Map<string, TypeValue>();
for (const [name, type] of args) {
bindings.set(base.env.get(name)!.key, type);
}
plan.base = { plan: base, bindings };
this.scan(target.node.getSourceFile());
}
}
}
}
return plan;
}
protobufFunction(
expression: ts.Expression,
source: ts.SourceFile,
): Requirement['fn'] | undefined {
let name: string | undefined;
if (ts.isIdentifier(expression)) {
const declaration = this.context.lookup(expression.text, expression, source);
name = declaration ? runtimeImportName(declaration.node) : undefined;
} else if (
ts.isPropertyAccessExpression(expression) &&
ts.isIdentifier(expression.expression)
) {
const declaration = this.context.lookup(expression.expression.text, expression, source);
if (
declaration &&
ts.isNamespaceImport(declaration.node) &&
(declaration.node.parent.parent.moduleSpecifier as ts.StringLiteral).text ===
packageName
) {
name = expression.name.text;
}
}
return name === 'protobuf_encode' || name === 'protobuf_decode' ? name : undefined;
}
parameters(node: Owner): readonly ts.ParameterDeclaration[] | undefined {
const parameters = ts.isClassDeclaration(node)
? (constructorOf(node)?.parameters ??
(this.plan(node).base ? this.parameters(this.plan(node).base!.plan.node) : undefined))
: node.parameters;
return parameters?.filter(
(param) => !ts.isIdentifier(param.name) || param.name.text !== 'this',
);
}
parameterName(plan: Plan, key: string): string {
let name = plan.names.get(key);
if (!name) {
name = unique(
plan.node.getSourceFile(),
`__pb_codec_${plan.node.pos}_${plan.names.size}`.replace(/-/g, '_'),
);
plan.names.set(key, name);
}
return name;
}
}
@@ -0,0 +1,169 @@
import ts from 'typescript';
import { fail } from '../source/context.js';
import type { TypeEnvironment } from '../schema/types.js';
import { memberType, type Signature } from './signatures.js';
import { callable } from './model.js';
import type { CallAnalysis } from './analysis.js';
interface Expected {
type: ts.TypeNode;
env: TypeEnvironment;
}
export class CallbackTypes {
constructor(private readonly graph: CallAnalysis) {}
context(
node: ts.Expression,
env: TypeEnvironment,
): { signature?: Signature; types?: ts.TypeNode[] } | undefined {
const expected = this.expected(node, env);
const signature = expected && this.graph.targets.signature(expected.type, expected.env);
if (signature) {
return { signature };
}
const parent = node.parent;
if (
ts.isCallExpression(parent) &&
parent.arguments[0] === node &&
ts.isPropertyAccessExpression(parent.expression) &&
['map', 'forEach', 'flatMap', 'filter', 'find', 'some', 'every'].includes(
parent.expression.name.text,
)
) {
const array = this.graph.targets.infer(
parent.expression.expression,
node.getSourceFile(),
);
const element = array && this.element(array);
if (element) {
return { types: [element] };
}
}
return;
}
private expected(node: ts.Node, env: TypeEnvironment): Expected | undefined {
const graph = this.graph;
let child = node;
if (
ts.isPropertyAccessExpression(child.parent) &&
child.parent.name.text === 'bind' &&
ts.isCallExpression(child.parent.parent)
) {
child = child.parent.parent;
}
while (
ts.isParenthesizedExpression(child.parent) ||
ts.isConditionalExpression(child.parent)
) {
child = child.parent;
}
const parent = child.parent;
if (
(ts.isVariableDeclaration(parent) ||
ts.isParameter(parent) ||
ts.isPropertyDeclaration(parent) ||
ts.isAsExpression(parent) ||
ts.isSatisfiesExpression(parent)) &&
parent.type
) {
return { type: parent.type, env };
}
if (
ts.isBinaryExpression(parent) &&
parent.right === child &&
parent.operatorToken.kind === ts.SyntaxKind.EqualsToken
) {
const type = graph.targets.infer(parent.left, node.getSourceFile());
if (type) {
return { type, env };
}
}
if (
(ts.isPropertyAssignment(parent) || ts.isShorthandPropertyAssignment(parent)) &&
ts.isObjectLiteralExpression(parent.parent)
) {
const object = this.expected(parent.parent, env);
return (
object &&
memberType(
object.type,
parent.name.getText().replace(/^['"]|['"]$/g, ''),
object.env,
graph.context,
graph.resolver,
)
);
}
if (ts.isArrayLiteralExpression(parent)) {
const array = this.expected(parent, env);
const type = array && this.element(array.type);
return type && array ? { type, env: array.env } : undefined;
}
if (ts.isReturnStatement(parent) || (ts.isArrowFunction(parent) && parent.body === child)) {
for (let fn: ts.Node | undefined = parent; fn; fn = fn.parent) {
if (callable(fn)) {
return fn.type ? { type: fn.type, env } : undefined;
}
}
}
if (ts.isCallExpression(parent)) {
const index = parent.arguments.indexOf(child as ts.Expression);
const site = graph.sites
.get(node.getSourceFile())
?.find((site) => site.node === parent);
const parameter = site?.target && graph.parameters(site.target.node)?.[index];
if (site?.target && parameter?.type) {
const bindings = graph.bindings(site, true);
const local = new Map(
[...site.target.env].map(([name, value]) => [
name,
graph.resolver.substitute(value, bindings),
]),
);
const signature = graph.targets.signature(parameter.type, local);
if (
signature?.node.typeParameters?.length &&
(site.target.node.getSourceFile().isDeclarationFile ||
(ts.isFunctionDeclaration(site.target.node) && !site.target.node.body))
) {
fail(
node,
'Polymorphic callbacks require the receiving implementation to participate in the build.',
);
}
return { type: parameter.type, env: local };
}
}
return;
}
private element(type: ts.TypeNode): ts.TypeNode | undefined {
return ts.isArrayTypeNode(type)
? type.elementType
: ts.isTypeReferenceNode(type) &&
['Array', 'ReadonlyArray'].includes(type.typeName.getText())
? type.typeArguments?.[0]
: undefined;
}
}
+323
View File
@@ -0,0 +1,323 @@
import ts from 'typescript';
import { fail } from '../source/context.js';
import type { TypeEnvironment, TypeValue } from '../schema/types.js';
import type { CallAnalysis } from './analysis.js';
import { callable, type Plan, type Requirement, type Edit } from './model.js';
import { codecIndex, unique } from './edits.js';
import { CallbackTypes } from './callback-types.js';
interface Adapter {
node: ts.Expression;
target: Plan;
bindings: Map<string, TypeValue>;
owner?: Plan;
signature?: Plan;
}
export class Callbacks {
readonly adapters = new Map<ts.Expression, Adapter>();
private readonly candidates: ts.Expression[] = [];
private cursor = 0;
private readonly types: CallbackTypes;
constructor(private readonly graph: CallAnalysis) {
this.types = new CallbackTypes(graph);
}
record(node: ts.Node): void {
switch (node.kind) {
case ts.SyntaxKind.ExpressionWithTypeArguments:
if (!ts.isHeritageClause(node.parent)) {
this.candidates.push(node as ts.ExpressionWithTypeArguments);
}
break;
case ts.SyntaxKind.VariableDeclaration:
case ts.SyntaxKind.Parameter:
case ts.SyntaxKind.PropertyDeclaration:
case ts.SyntaxKind.PropertyAssignment:
this.candidate((node as ts.VariableDeclaration).initializer);
break;
case ts.SyntaxKind.CallExpression:
for (const argument of (node as ts.CallExpression).arguments) {
this.candidate(argument);
}
break;
case ts.SyntaxKind.ArrayLiteralExpression:
for (const element of (node as ts.ArrayLiteralExpression).elements) {
this.candidate(element);
}
break;
case ts.SyntaxKind.ShorthandPropertyAssignment:
this.candidate((node as ts.ShorthandPropertyAssignment).name);
break;
case ts.SyntaxKind.ReturnStatement:
case ts.SyntaxKind.ParenthesizedExpression:
case ts.SyntaxKind.AsExpression:
case ts.SyntaxKind.SatisfiesExpression:
this.candidate((node as ts.ReturnStatement).expression);
break;
case ts.SyntaxKind.ConditionalExpression: {
const expression = node as ts.ConditionalExpression;
this.candidate(expression.whenTrue);
this.candidate(expression.whenFalse);
break;
}
case ts.SyntaxKind.BinaryExpression: {
const expression = node as ts.BinaryExpression;
if (expression.operatorToken.kind === ts.SyntaxKind.EqualsToken) {
this.candidate(expression.right);
}
break;
}
case ts.SyntaxKind.PropertyAccessExpression: {
const expression = node as ts.PropertyAccessExpression;
if (expression.name.text === 'bind') {
this.candidate(expression.expression);
}
}
}
}
private candidate(node: ts.Expression | undefined): void {
if (
node &&
(ts.isIdentifier(node) ||
ts.isPropertyAccessExpression(node) ||
ts.isElementAccessExpression(node) ||
callable(node))
) {
this.candidates.push(node);
}
}
collect(): void {
while (this.cursor < this.candidates.length) {
this.add(this.candidates[this.cursor++]);
}
}
private add(node: ts.Expression): void {
const graph = this.graph;
const env = graph.environment(node);
const explicit = ts.isExpressionWithTypeArguments(node) ? node : undefined;
const target = graph.targets.target(
explicit?.expression ?? node,
node.getSourceFile(),
env,
);
if (!target || ts.isClassDeclaration(target.node)) {
return;
}
const plan = graph.plan(target.node);
if (!plan.node.typeParameters?.length) {
return;
}
graph.scan(plan.node.getSourceFile());
const ownerNode = graph.owner(node);
const adapter: Adapter = {
node,
target: plan,
bindings: new Map(target.bindings),
owner: ownerNode ? graph.plan(ownerNode) : undefined,
};
if (explicit) {
const args = graph.resolver.arguments(
plan.node.typeParameters,
explicit.typeArguments,
node.getSourceFile(),
env,
plan.node,
);
for (const [name, type] of args) {
adapter.bindings.set(plan.env.get(name)!.key, type);
}
} else {
const context = this.types.context(node, env);
if (!context) {
return;
}
const signature = context.signature;
const local = new Map(signature?.env ?? env);
if (signature?.node.typeParameters?.length) {
adapter.signature = graph.plan(signature.node);
for (const param of signature.node.typeParameters) {
local.set(param.name.text, adapter.signature.env.get(param.name.text)!);
}
}
const parameters = graph.parameters(plan.node) ?? [];
const actual =
context.types ?? graph.parameters(signature!.node)!.map((param) => param.type);
parameters.forEach((param, index) => {
if (
param.type &&
actual[index] &&
plan.runtime !== 'protobuf_decode' &&
mentionsGeneric(param.type, plan.env)
) {
graph.unify(
graph.resolver.type(param.type, param.getSourceFile(), plan.env),
graph.resolver.type(actual[index]!, actual[index]!.getSourceFile(), local),
adapter.bindings,
node,
);
}
});
if (plan.runtime === 'protobuf_decode' && signature) {
adapter.bindings.set(
plan.env.get('T')!.key,
graph.resolver.type(
signature.node.type!,
signature.node.getSourceFile(),
local,
),
);
}
for (const parameter of plan.node.typeParameters ?? []) {
const key = plan.env.get(parameter.name.text)!.key;
if (!adapter.bindings.has(key) && parameter.default) {
adapter.bindings.set(
key,
graph.resolver.type(parameter.default, parameter.getSourceFile(), env),
);
}
}
}
this.adapters.set(node, adapter);
}
connections(): { target: Plan; owner?: Plan; bindings: () => Map<string, TypeValue> }[] {
return [...this.adapters.values()].map((adapter) => ({
target: adapter.target,
owner: adapter.signature ?? adapter.owner,
bindings: () => adapter.bindings,
}));
}
emit(
source: ts.SourceFile,
reference: (req: Requirement, owner?: Plan) => string,
edits: Edit[],
): void {
for (const adapter of this.adapters.values()) {
if (adapter.node.getSourceFile() !== source || !adapter.target.requirements.size) {
continue;
}
const { target, signature, node } = adapter;
if (target === signature) {
continue;
}
const requirements = [...target.requirements.values()].map((req) =>
this.graph.instantiate(req, adapter.bindings),
);
for (const req of requirements) {
if (
[...target.env.values()].some(
(type) => type.dynamic && req.type.key === type.key,
)
) {
fail(
node,
'Cannot infer callback schema; annotate its parameter type or supply type arguments.',
);
}
}
const codecs = requirements.map((req) => reference(req, signature ?? adapter.owner));
if (target.runtime && !signature) {
const text =
target.runtime === 'protobuf_decode'
? `(bytes => ${codecs[0]}(bytes))`
: codecs[0];
edits.push({ start: node.getStart(source), end: node.end, text });
continue;
}
const params = this.graph.parameters(target.node) ?? [];
const count = params.findIndex((param) => !!param.dotDotDotToken);
const arity = count < 0 ? params.length : count;
const signatureParams = signature && this.graph.parameters(signature.node)!;
const signatureRest =
signatureParams?.findIndex((param) => !!param.dotDotDotToken) ?? -1;
const incomingArity = signatureParams
? signatureRest < 0
? signatureParams.length
: signatureRest
: arity;
const prefix = unique(source, '__pb_callback');
const args = Array.from(
{ length: Math.max(arity, incomingArity) },
(_, index) => `${prefix}_${index}`,
);
const hidden = signature
? [...signature.requirements.keys()].map((key) =>
this.graph.parameterName(signature, key),
)
: [];
const incoming = args.map((name, index) =>
params[index]?.initializer ? `${name} = undefined` : name,
);
incoming.splice(incomingArity, 0, ...hidden.map((name) => `${name} = undefined`));
if (count >= 0) {
incoming.push(`...${prefix}_rest`);
}
const index = codecIndex(params);
const forwarded = [...args.slice(0, index), ...codecs, ...args.slice(index, arity)];
if (count >= 0) {
forwarded.push(...args.slice(arity), `...${prefix}_rest`);
}
const call = target.runtime
? `${codecs[0]}(${args[0]})`
: `${prefix}_fn.call(this, ${forwarded.join(', ')})`;
const property = ts.isShorthandPropertyAssignment(node.parent)
? node.getText(source) + ': '
: '';
edits.push({
start: node.getStart(source),
end: node.getStart(source),
text: `${property}((${prefix}_fn) => function(${incoming.join(', ')}) { return ${call}; })(`,
});
edits.push({ start: node.end, end: node.end, text: ')', order: -1 });
}
}
}
export function mentionsGeneric(node: ts.Node, env: TypeEnvironment): boolean {
if (ts.isIdentifier(node) && env.get(node.text)?.dynamic) {
return true;
}
return ts.forEachChild(node, (child) => mentionsGeneric(child, env)) ?? false;
}
+326
View File
@@ -0,0 +1,326 @@
import { codecIndex, insertArguments, insertParameters, unique } from './edits.js';
import ts from 'typescript';
import { SourceContext, fail } from '../source/context.js';
import { SchemaResolver } from '../schema/resolver.js';
import { generateCode } from '../codegen/generator.js';
import { constructorOf, type Plan, type Requirement, type Edit } from './model.js';
import { CallAnalysis } from './analysis.js';
import { validateSource } from './validate.js';
/** Generic calls receive generated codec functions, preserving the original body and closure. */
export class ProtobufCompiler {
readonly context = new SourceContext();
private resolver = new SchemaResolver(this.context);
private graph = new CallAnalysis(this.context, this.resolver);
private output = new Map<string, { source: string; code: string | null }>();
private entries = new Set<string>();
clear(): void {
this.reset();
this.entries.clear();
}
private reset(): void {
this.context.clear();
this.output.clear();
this.resolver = new SchemaResolver(this.context);
this.graph = new CallAnalysis(this.context, this.resolver);
}
prepare(files: Iterable<string>): void {
for (const file of files) {
this.entries.add(file);
}
try {
for (const file of this.entries) {
this.graph.scan(this.context.source(file));
}
this.graph.callbacks.collect();
this.graph.propagate();
} catch (error) {
this.reset();
throw error;
}
}
compile(code: string, file: string): string | null {
try {
return this.transform(code, file);
} catch (error) {
this.reset();
throw error;
}
}
private transform(code: string, file: string): string | null {
const cached = this.output.get(file);
if (cached?.source === code) {
return cached.code;
}
const revision = this.context.revision;
const source = this.context.source(file, code);
if (!this.graph.sites.size && this.entries.size) {
this.prepare(this.entries);
}
if (cached || this.context.revision !== revision) {
const entries = [...this.entries];
this.clear();
this.context.files.set(source.fileName, source);
this.prepare(entries);
}
this.graph.scan(source);
this.graph.callbacks.collect();
this.graph.propagate();
validateSource(this.graph, source);
const edits: Edit[] = [];
const encodeRoots = new Set<string>();
const decodeRoots = new Set<string>();
const reference = (requirement: Requirement, owner?: Plan, constructor = false): string => {
if (requirement.type.dynamic) {
if (!owner) {
fail(
requirement.node,
'Unresolved generic codec outside a generic function or class.',
);
}
const name = this.graph.parameterName(owner, requirement.key);
return ts.isClassDeclaration(owner.node) && !constructor ? `this.#${name}` : name;
}
const name = this.resolver.message(requirement.type, requirement.node);
(requirement.fn === 'protobuf_encode' ? encodeRoots : decodeRoots).add(name);
return `${requirement.fn}_${name}`;
};
for (const site of this.graph.sites.get(source) ?? []) {
let constructor = false;
for (
let parent: ts.Node | undefined = site.node;
parent && parent !== site.owner?.node;
parent = parent.parent
) {
if (ts.isConstructorDeclaration(parent)) {
constructor = true;
break;
}
}
if (site.direct) {
edits.push({
start: site.node.expression.getStart(source),
end: site.node.typeArguments
? site.node.typeArguments.end + 1
: site.node.expression.end,
text: reference(site.direct, site.owner, constructor),
});
} else if (site.target?.requirements.size) {
const args = [...site.target.requirements.values()].map((req) =>
reference(
this.graph.instantiate(req, this.graph.bindings(site)),
site.owner,
constructor,
),
);
insertArguments(
site.node,
this.graph.parameters(site.target.node),
args,
edits,
source,
);
}
}
this.graph.callbacks.emit(source, reference, edits);
for (const plan of this.graph.plans.values()) {
if (
ts.isTypeNode(plan.node) ||
ts.isCallSignatureDeclaration(plan.node) ||
plan.node.getSourceFile() !== source ||
(plan.requirements.size === 0 && !plan.base?.plan.requirements.size)
) {
continue;
}
const names = [...plan.requirements.keys()].map((key) =>
this.graph.parameterName(plan, key),
);
if (ts.isClassDeclaration(plan.node)) {
const constructor = constructorOf(plan.node);
const assignments = names.map((name) => `this.#${name} = ${name};`).join('\n');
if (names.length) {
edits.push({
start: plan.node.members.pos,
end: plan.node.members.pos,
text: '\n' + names.map((name) => `#${name};`).join('\n') + '\n',
});
}
if (constructor?.body) {
if (names.length) {
insertParameters(constructor.parameters, names, edits, source);
}
const superStatement = constructor.body.statements.find(
(statement) =>
ts.isExpressionStatement(statement) &&
ts.isCallExpression(statement.expression) &&
statement.expression.expression.kind === ts.SyntaxKind.SuperKeyword,
);
if (plan.base && !superStatement) {
fail(
constructor,
'Generic derived constructors require a top-level super(...) call.',
);
}
const position = superStatement?.end ?? constructor.body.getStart(source) + 1;
edits.push({ start: position, end: position, text: assignments });
} else if (plan.base) {
const inherited = this.graph.parameters(plan.base.plan.node);
const count = codecIndex(inherited);
const argsName = unique(source, '__pb_constructor_args');
const locals = names
.map((name, i) => `const ${name} = ${argsName}[${count + i}];`)
.join('\n');
const codecs = [...plan.base.plan.requirements.values()].map((req) =>
reference(this.graph.instantiate(req, plan.base!.bindings), plan, true),
);
const forwarded = [
`...${argsName}.slice(0, ${count})`,
...codecs,
`...${argsName}.slice(${count + names.length})`,
];
edits.push({
start: plan.node.members.pos,
end: plan.node.members.pos,
text: `\nconstructor(...${argsName}) { ${locals}\nsuper(${forwarded.join(', ')});\n${assignments} }\n`,
});
} else {
edits.push({
start: plan.node.members.pos,
end: plan.node.members.pos,
text: `\nconstructor(${names.map((name) => name + ' = undefined').join(', ')}) { ${assignments} }\n`,
});
}
} else {
insertParameters(plan.node.parameters, names, edits, source);
}
}
if (!edits.length) {
this.output.set(file, { source: code, code: null });
return null;
}
edits.sort((a, b) => a.start - b.start || (a.order ?? 0) - (b.order ?? 0) || a.end - b.end);
const parts: string[] = [];
let offset = 0;
for (const edit of edits) {
if (edit.start < offset) {
throw new Error('Overlapping protobuf compiler edits.');
}
offset = edit.end;
}
let generated = generateCode(this.resolver.registry, {
encodeRoots,
decodeRoots,
});
// Keep generated identifiers out of the user's namespace, including helper names.
const prefix = unique(source, '__pb_generated_');
const generatedNames = new Set(
[
...generated.matchAll(
/\b(?:function|const|let)\s+((?:protobuf_(?:encode|decode)_|__)[\p{ID_Continue}$]+)/gu,
),
].map((match) => match[1]),
);
const hasCollision = [...generatedNames].some((name) => source.text.includes(name));
const renamed = (text: string): string => {
if (!hasCollision) {
return text;
}
const scanner = ts.createScanner(
ts.ScriptTarget.Latest,
false,
ts.LanguageVariant.Standard,
text,
);
const chunks: string[] = [];
let position = 0;
for (
let token = scanner.scan();
token !== ts.SyntaxKind.EndOfFileToken;
token = scanner.scan()
) {
if (
token !== ts.SyntaxKind.Identifier ||
!generatedNames.has(scanner.getTokenText())
) {
continue;
}
chunks.push(
text.slice(position, scanner.getTokenPos()),
prefix + scanner.getTokenText(),
);
position = scanner.getTextPos();
}
return chunks.join('') + text.slice(position);
};
generated = renamed(generated);
// Rename only inserted codec references, never arbitrary source strings or user identifiers.
for (const edit of edits) {
edit.text = renamed(edit.text);
}
parts.length = 0;
offset = 0;
for (const edit of edits) {
parts.push(code.slice(offset, edit.start), edit.text);
offset = edit.end;
}
parts.push(code.slice(offset));
const rewritten = parts.join('');
let preambleAt = code.startsWith('#!') ? code.indexOf('\n') + 1 : 0;
for (const statement of source.statements) {
if (!ts.isExpressionStatement(statement) || !ts.isStringLiteral(statement.expression)) {
break;
}
preambleAt = statement.end;
}
const transformed =
rewritten.slice(0, preambleAt) + '\n' + generated + '\n' + rewritten.slice(preambleAt);
this.output.set(file, { source: code, code: transformed });
return transformed;
}
}
+80
View File
@@ -0,0 +1,80 @@
import ts from 'typescript';
import type { Edit, Site } from './model.js';
export function unique(source: ts.SourceFile, base: string): string {
while (source.text.includes(base)) {
base += '_';
}
return base;
}
export function codecIndex(params: readonly ts.ParameterDeclaration[] = []): number {
const index = params.findIndex((param) => !!param.initializer || !!param.dotDotDotToken);
return index < 0 ? params.length : index;
}
export function insertParameters(
params: ts.NodeArray<ts.ParameterDeclaration>,
names: string[],
edits: Edit[],
source: ts.SourceFile,
): void {
names = names.map((name) => name + ' = undefined');
const index = codecIndex(params);
if (index < params.length) {
edits.push({
start: params[index].getStart(source),
end: params[index].getStart(source),
text: names.join(', ') + ', ',
});
} else {
edits.push({
start: params.end,
end: params.end,
text: (params.length && !params.hasTrailingComma ? ', ' : '') + names.join(', '),
});
}
}
export function insertArguments(
node: Site['node'],
params: readonly ts.ParameterDeclaration[] | undefined,
args: string[],
edits: Edit[],
source: ts.SourceFile,
): void {
const index = codecIndex(params);
const actual = node.arguments;
if (actual?.some(ts.isSpreadElement)) {
const values = unique(source, '__pb_arguments');
edits.push({
start: actual[0].getStart(source),
end: actual[0].getStart(source),
text: `...((${values}) => { while (${values}.length < ${index}) ${values}.push(undefined); ${values}.splice(${index}, 0, ${args.join(', ')}); return ${values}; })([`,
});
edits.push({ start: actual.end, end: actual.end, text: '])', order: 1 });
return;
}
const missing = Math.max(0, index - (actual?.length ?? 0));
const text = [...Array(missing).fill('undefined'), ...args].join(', ');
if (!actual) {
edits.push({ start: node.end, end: node.end, text: `(${text})` });
} else if (actual.length > index) {
edits.push({
start: actual[index].getStart(source),
end: actual[index].getStart(source),
text: text + ', ',
});
} else {
edits.push({
start: actual.end,
end: actual.end,
text: (actual.length && !actual.hasTrailingComma ? ', ' : '') + text,
});
}
}
+99
View File
@@ -0,0 +1,99 @@
import ts from 'typescript';
import type { TypeEnvironment, TypeValue } from '../schema/types.js';
export type Callable =
| ts.FunctionDeclaration
| ts.FunctionExpression
| ts.ArrowFunction
| ts.MethodDeclaration
| ts.FunctionTypeNode
| ts.CallSignatureDeclaration;
export type Owner = Callable | ts.ClassDeclaration;
export interface Requirement {
key: string;
fn: 'protobuf_encode' | 'protobuf_decode';
type: TypeValue;
node: ts.Node;
}
export interface Plan {
runtime?: Requirement['fn'];
node: Owner;
env: TypeEnvironment;
requirements: Map<string, Requirement>;
names: Map<string, string>;
base?: { plan: Plan; bindings: Map<string, TypeValue> };
}
export interface Target {
node: Owner;
bindings: Map<string, TypeValue>;
}
export interface Site {
node: ts.CallExpression | ts.NewExpression;
owner?: Plan;
direct?: Requirement;
target?: Plan;
bindings?: Map<string, TypeValue>;
typeArgs?: readonly ts.TypeNode[];
env?: TypeEnvironment;
bound?: boolean;
}
export interface Edit {
order?: number;
start: number;
end: number;
text: string;
}
export function callable(node: ts.Node): node is Callable {
switch (node.kind) {
case ts.SyntaxKind.FunctionDeclaration:
case ts.SyntaxKind.FunctionExpression:
case ts.SyntaxKind.ArrowFunction:
case ts.SyntaxKind.MethodDeclaration:
case ts.SyntaxKind.FunctionType:
case ts.SyntaxKind.CallSignature:
return true;
default:
return false;
}
}
export function constructorOf(node: ts.ClassDeclaration): ts.ConstructorDeclaration | undefined {
return node.members.find(
(member): member is ts.ConstructorDeclaration =>
ts.isConstructorDeclaration(member) && !!member.body,
);
}
export function memberName(name: ts.PropertyName): string | undefined {
if (ts.isIdentifier(name) || ts.isStringLiteralLike(name) || ts.isNumericLiteral(name)) {
return name.text;
}
if (
ts.isComputedPropertyName(name) &&
(ts.isStringLiteralLike(name.expression) || ts.isNumericLiteral(name.expression))
) {
return name.expression.text;
}
return;
}
export function access(
expression: ts.Expression,
): { receiver: ts.Expression; name: string } | undefined {
if (ts.isPropertyAccessExpression(expression)) {
return { receiver: expression.expression, name: expression.name.text };
}
if (
ts.isElementAccessExpression(expression) &&
(ts.isStringLiteralLike(expression.argumentExpression) ||
ts.isNumericLiteral(expression.argumentExpression))
) {
return { receiver: expression.expression, name: expression.argumentExpression.text };
}
return;
}
+404
View File
@@ -0,0 +1,404 @@
import ts from 'typescript';
import { fail } from '../source/context.js';
import type { TypeEnvironment } from '../schema/types.js';
import {
access,
callable,
constructorOf,
memberName,
type Callable,
type Target,
} from './model.js';
import type { CallTargets } from './targets.js';
interface Receiver {
node: ts.ClassDeclaration | ts.ObjectLiteralExpression;
bindings: Target['bindings'];
env: TypeEnvironment;
}
type Returning = Callable | ts.GetAccessorDeclaration;
export class Receivers {
private readonly returns = new WeakMap<Returning, ts.Expression[]>();
constructor(private readonly targets: CallTargets) {}
resolve(
expression: ts.Expression,
source: ts.SourceFile,
env: TypeEnvironment,
seen = new Set<ts.Node>(),
): Receiver | undefined {
if (seen.has(expression)) {
return;
}
seen.add(expression);
const { context } = this.targets;
if (
ts.isParenthesizedExpression(expression) ||
ts.isNonNullExpression(expression) ||
ts.isSatisfiesExpression(expression)
) {
return this.resolve(expression.expression, source, env, seen);
}
if (ts.isAsExpression(expression) || ts.isTypeAssertionExpression(expression)) {
return (
this.annotation(expression.type, env, seen) ??
this.resolve(expression.expression, source, env, seen)
);
}
if (ts.isObjectLiteralExpression(expression)) {
return { node: expression, env, bindings: new Map() };
}
if (expression.kind === ts.SyntaxKind.ThisKeyword) {
for (let parent = expression.parent; parent; parent = parent.parent) {
if (ts.isClassDeclaration(parent)) {
const target = this.targets.capture(
expression,
{ node: parent, bindings: new Map() },
env,
);
return {
node: parent,
bindings: target.bindings,
env: this.targets.environment(target),
};
}
}
}
if (ts.isIdentifier(expression)) {
const declaration = context.lookup(expression.text, expression, source)?.node;
if (declaration && ts.isClassDeclaration(declaration)) {
return {
node: declaration,
bindings: new Map(),
env: this.targets.graph.plan(declaration).env,
};
}
if (
declaration &&
(ts.isVariableDeclaration(declaration) || ts.isParameter(declaration))
) {
const annotated = declaration.type && this.annotation(declaration.type, env, seen);
if (annotated) {
return annotated;
}
if (declaration.initializer) {
return this.resolve(
declaration.initializer,
declaration.getSourceFile(),
env,
seen,
);
}
}
}
const property = access(expression);
if (property) {
const receiver = this.resolve(property.receiver, source, env, seen);
const member = receiver && this.member(receiver, property.name);
if (!member) {
return;
}
const node = member.node;
if ('type' in node && node.type) {
const annotated = this.annotation(node.type, member.env, seen);
if (annotated) {
return annotated;
}
}
if (ts.isGetAccessorDeclaration(node)) {
return this.returned(node, member.env, seen, expression);
}
if ('initializer' in node && node.initializer) {
return this.resolve(node.initializer, node.getSourceFile(), member.env, seen);
}
if (ts.isShorthandPropertyAssignment(node)) {
return this.resolve(node.name, node.getSourceFile(), member.env, seen);
}
}
if (ts.isNewExpression(expression) || ts.isCallExpression(expression)) {
const target = this.targets.target(expression.expression, source, env, seen);
if (!target) {
return;
}
const specialized = this.targets.specialize(expression, target, env);
const local = this.targets.environment(specialized);
if (ts.isNewExpression(expression)) {
return ts.isClassDeclaration(target.node)
? { node: target.node, bindings: specialized.bindings, env: local }
: undefined;
}
if (!callable(target.node)) {
return;
}
const annotated = target.node.type && this.annotation(target.node.type, local, seen);
return annotated || this.returned(target.node, local, seen, expression);
}
if (ts.isConditionalExpression(expression)) {
return this.merge([expression.whenTrue, expression.whenFalse], env, seen, expression);
}
return;
}
member(
receiver: Receiver,
name: string,
):
| {
node: ts.ClassElement | ts.ObjectLiteralElementLike | ts.ParameterDeclaration;
bindings: Target['bindings'];
env: TypeEnvironment;
}
| undefined {
const { graph, resolver } = this.targets;
let node = receiver.node;
const bindings = new Map(receiver.bindings);
for (;;) {
const members = ts.isClassDeclaration(node) ? node.members : node.properties;
const member =
members.find(
(member) =>
member.name &&
memberName(member.name) === name &&
(!ts.isMethodDeclaration(member) || !!member.body),
) ??
(ts.isClassDeclaration(node)
? constructorOf(node)?.parameters.find(
(param) =>
ts.isIdentifier(param.name) &&
param.name.text === name &&
param.modifiers?.length,
)
: undefined);
if (member) {
const env = ts.isClassDeclaration(node)
? this.targets.environment({ node, bindings })
: receiver.env;
return { node: member, bindings, env };
}
const base = ts.isClassDeclaration(node) && graph.plan(node).base;
if (!base || !ts.isClassDeclaration(base.plan.node)) {
return;
}
for (const [key, type] of base.bindings) {
bindings.set(key, resolver.substitute(type, bindings));
}
node = base.plan.node;
}
}
private annotation(
type: ts.TypeNode,
env: TypeEnvironment,
seen: Set<ts.Node>,
): Receiver | undefined {
if (seen.has(type)) {
return;
}
seen.add(type);
if (ts.isParenthesizedTypeNode(type)) {
return this.annotation(type.type, env, seen);
}
if (ts.isUnionTypeNode(type)) {
const types = type.types.filter(
(item) =>
item.kind !== ts.SyntaxKind.UndefinedKeyword &&
!(
ts.isLiteralTypeNode(item) &&
item.literal.kind === ts.SyntaxKind.NullKeyword
),
);
if (types.length === 1) {
return this.annotation(types[0], env, seen);
}
return;
}
if (!ts.isTypeReferenceNode(type)) {
return;
}
const { context, resolver, graph } = this.targets;
const ref = context.typeName(type.typeName, type.getSourceFile());
if (!ref || (!ts.isClassDeclaration(ref.node) && !ts.isTypeAliasDeclaration(ref.node))) {
return;
}
if (ts.isTypeAliasDeclaration(ref.node) && !this.classShape(ref.node.type)) {
return;
}
const args = resolver.arguments(
ref.node.typeParameters,
type.typeArguments,
type.getSourceFile(),
env,
ref.node,
);
if (ts.isTypeAliasDeclaration(ref.node)) {
return this.annotation(ref.node.type, new Map([...env, ...args]), seen);
}
const plan = graph.plan(ref.node);
const target = this.targets.capture(
type,
{
node: ref.node,
bindings: new Map([...args].map(([name, type]) => [plan.env.get(name)!.key, type])),
},
env,
);
return { node: ref.node, bindings: target.bindings, env: this.targets.environment(target) };
}
private classShape(type: ts.TypeNode, seen = new Set<ts.Node>()): boolean {
if (seen.has(type)) {
return false;
}
seen.add(type);
if (ts.isParenthesizedTypeNode(type)) {
return this.classShape(type.type, seen);
}
if (ts.isUnionTypeNode(type)) {
return type.types.some((member) => this.classShape(member, seen));
}
if (!ts.isTypeReferenceNode(type)) {
return false;
}
const node = this.targets.context.typeName(type.typeName, type.getSourceFile())?.node;
return (
!!node &&
(ts.isClassDeclaration(node) ||
(ts.isTypeAliasDeclaration(node) && this.classShape(node.type, seen)))
);
}
private returned(
node: Returning,
env: TypeEnvironment,
seen: Set<ts.Node>,
origin: ts.Node,
): Receiver | undefined {
const cached = this.returns.get(node);
if (cached) {
return this.merge(cached, env, seen, origin);
}
const expressions: ts.Expression[] = [];
if ('body' in node && node.body) {
if (ts.isBlock(node.body)) {
const visit = (child: ts.Node): void => {
if (ts.isFunctionLike(child) || ts.isClassDeclaration(child)) {
return;
}
if (ts.isReturnStatement(child)) {
if (child.expression) {
expressions.push(child.expression);
}
} else {
ts.forEachChild(child, visit);
}
};
visit(node.body);
} else {
expressions.push(node.body);
}
}
this.returns.set(node, expressions);
return this.merge(expressions, env, seen, origin);
}
private merge(
expressions: ts.Expression[],
env: TypeEnvironment,
seen: Set<ts.Node>,
origin: ts.Node,
): Receiver | undefined {
let result: Receiver | undefined;
for (const expression of expressions) {
const next = this.resolve(expression, expression.getSourceFile(), env, new Set(seen));
if (!next) {
return;
}
if (
result &&
(result.node !== next.node ||
[...result.env].some(([name, type]) => next.env.get(name)?.key !== type.key))
) {
// Data objects returned by ordinary functions need no codec
// specialization, even when their branches have different shapes.
if (
ts.isObjectLiteralExpression(result.node) &&
ts.isObjectLiteralExpression(next.node) &&
![...result.node.properties, ...next.node.properties].some(
(member) =>
callable(member) ||
(ts.isPropertyAssignment(member) && callable(member.initializer)),
)
) {
return;
}
fail(origin, 'Receiver branches must resolve to the same specialization.');
}
result = next;
}
return result;
}
}
+212
View File
@@ -0,0 +1,212 @@
import ts from 'typescript';
import type { SourceContext } from '../source/context.js';
import type { SchemaResolver } from '../schema/resolver.js';
import type { TypeEnvironment } from '../schema/types.js';
export interface Signature {
node: ts.FunctionTypeNode | ts.CallSignatureDeclaration;
env: TypeEnvironment;
}
export function memberType(
type: ts.TypeNode,
name: string,
env: TypeEnvironment,
context: SourceContext,
resolver: SchemaResolver,
seen = new Set<ts.Node>(),
): { type: ts.TypeNode; env: TypeEnvironment } | undefined {
if (seen.has(type)) {
return;
}
seen.add(type);
if (ts.isTypeLiteralNode(type)) {
const member = type.members.find(
(member) =>
member.name &&
(ts.isIdentifier(member.name) || ts.isStringLiteral(member.name)) &&
member.name.text === name,
);
return member && ts.isPropertySignature(member) && member.type
? { type: member.type, env }
: undefined;
}
if (ts.isTypeReferenceNode(type)) {
const ref = context.typeName(type.typeName, type.getSourceFile());
if (
!ref ||
(!ts.isTypeAliasDeclaration(ref.node) && !ts.isInterfaceDeclaration(ref.node))
) {
return;
}
if (ts.isTypeAliasDeclaration(ref.node) && !memberShape(ref.node.type, name, context)) {
return;
}
const local = new Map([
...env,
...resolver.arguments(
ref.node.typeParameters,
type.typeArguments,
type.getSourceFile(),
env,
ref.node,
true,
),
]);
if (ts.isTypeAliasDeclaration(ref.node)) {
return memberType(ref.node.type, name, local, context, resolver, seen);
}
const member = ref.node.members.find(
(member) => member.name?.getText(ref.source).replace(/^['"]|['"]$/g, '') === name,
);
return member && ts.isPropertySignature(member) && member.type
? { type: member.type, env: local }
: undefined;
}
return;
}
export function resolveSignature(
type: ts.TypeNode,
env: TypeEnvironment,
context: SourceContext,
resolver: SchemaResolver,
seen = new Set<ts.Node>(),
): Signature | undefined {
if (seen.has(type)) {
return;
}
seen.add(type);
if (ts.isParenthesizedTypeNode(type)) {
return resolveSignature(type.type, env, context, resolver, seen);
}
if (ts.isFunctionTypeNode(type)) {
return { node: type, env };
}
if (ts.isTypeLiteralNode(type)) {
const signatures = type.members.filter(ts.isCallSignatureDeclaration);
if (signatures.length === 1) {
return { node: signatures[0], env };
}
}
if (ts.isTypeReferenceNode(type)) {
const ref = context.typeName(type.typeName, type.getSourceFile());
if (!ref) {
return;
}
const node = ref.node;
if (!ts.isTypeAliasDeclaration(node) && !ts.isInterfaceDeclaration(node)) {
return;
}
if (!callableShape(type, context)) {
return;
}
const local = new Map([
...env,
...resolver.arguments(
node.typeParameters,
type.typeArguments,
type.getSourceFile(),
env,
node,
true,
),
]);
return ts.isTypeAliasDeclaration(node)
? resolveSignature(node.type, local, context, resolver, seen)
: { node: node.members.find(ts.isCallSignatureDeclaration)!, env: local };
}
return;
}
export function callableShape(
type: ts.TypeNode,
context: SourceContext,
seen = new Set<ts.Node>(),
): boolean {
if (seen.has(type)) {
return false;
}
seen.add(type);
if (ts.isFunctionTypeNode(type)) {
return true;
}
if (ts.isParenthesizedTypeNode(type)) {
return callableShape(type.type, context, seen);
}
if (ts.isTypeLiteralNode(type)) {
return type.members.filter(ts.isCallSignatureDeclaration).length === 1;
}
if (!ts.isTypeReferenceNode(type)) {
return false;
}
const node = context.typeName(type.typeName, type.getSourceFile())?.node;
return (
!!node &&
(ts.isTypeAliasDeclaration(node)
? callableShape(node.type, context, seen)
: ts.isInterfaceDeclaration(node) &&
node.members.filter(ts.isCallSignatureDeclaration).length === 1)
);
}
function memberShape(
type: ts.TypeNode,
name: string,
context: SourceContext,
seen = new Set<ts.Node>(),
): boolean {
if (seen.has(type)) {
return false;
}
seen.add(type);
if (ts.isTypeLiteralNode(type)) {
return type.members.some(
(member) => member.name?.getText().replace(/^['"]|['"]$/g, '') === name,
);
}
if (!ts.isTypeReferenceNode(type)) {
return false;
}
const node = context.typeName(type.typeName, type.getSourceFile())?.node;
return (
!!node &&
(ts.isTypeAliasDeclaration(node)
? memberShape(node.type, name, context, seen)
: ts.isInterfaceDeclaration(node) &&
node.members.some(
(member) => member.name?.getText().replace(/^['"]|['"]$/g, '') === name,
))
);
}
+345
View File
@@ -0,0 +1,345 @@
import ts from 'typescript';
import { packageName } from '../package.js';
import { SourceContext, runtimeImportName, type DeclarationRef } from '../source/context.js';
import { SchemaResolver } from '../schema/resolver.js';
import type { TypeEnvironment, TypeValue } from '../schema/types.js';
import { access, callable, type Plan, type Target, type Requirement, type Site } from './model.js';
import { Receivers } from './receivers.js';
import { callableShape, memberType, resolveSignature, type Signature } from './signatures.js';
interface Plans {
environment(node: ts.Node): TypeEnvironment;
plan(node: Target['node']): Plan;
runtimePlan(fn: Requirement['fn']): Plan;
bindings(site: Site, partial?: boolean): Map<string, TypeValue>;
}
export class CallTargets {
private readonly receivers = new Receivers(this);
constructor(
readonly context: SourceContext,
readonly resolver: SchemaResolver,
readonly graph: Plans,
) {}
signature(type: ts.TypeNode, env: TypeEnvironment) {
return resolveSignature(type, env, this.context, this.resolver);
}
signatureTarget(signature: Signature): Target {
const plan = this.graph.plan(signature.node);
return {
node: signature.node,
bindings: new Map(
[...signature.env].flatMap(([name, type]) => {
if (signature.node.typeParameters?.some((param) => param.name.text === name)) {
return [];
}
const parameter = plan.env.get(name);
return parameter ? [[parameter.key, type] as const] : [];
}),
),
};
}
target(
expression: ts.Expression,
source: ts.SourceFile,
env: TypeEnvironment,
seen = new Set<ts.Node>(),
): Target | undefined {
if (seen.has(expression)) {
return;
}
seen.add(expression);
if (callable(expression)) {
return { node: expression, bindings: new Map() };
}
if (ts.isCallExpression(expression)) {
const factory = this.target(expression.expression, source, env, seen);
if (
factory &&
callable(factory.node) &&
factory.node.type &&
callableShape(factory.node.type, this.context)
) {
const local = this.environment(this.specialize(expression, factory, env));
const signature = this.signature(factory.node.type, local);
if (signature) {
return this.signatureTarget(signature);
}
}
return;
}
if (
ts.isParenthesizedExpression(expression) ||
ts.isAsExpression(expression) ||
ts.isNonNullExpression(expression)
) {
return this.target(expression.expression, source, env, seen);
}
if (expression.kind === ts.SyntaxKind.SuperKeyword) {
for (let parent: ts.Node | undefined = expression; parent; parent = parent.parent) {
if (ts.isClassDeclaration(parent)) {
const base = this.graph.plan(parent).base;
return base
? { node: base.plan.node, bindings: new Map(base.bindings) }
: undefined;
}
}
}
let ref: DeclarationRef | undefined;
const property = access(expression);
if (ts.isIdentifier(expression)) {
ref = this.context.lookup(expression.text, expression, source);
} else if (property) {
let receiver: ts.Expression = property.receiver;
while (ts.isParenthesizedExpression(receiver)) {
receiver = receiver.expression;
}
const declaration = ts.isIdentifier(receiver)
? this.context.lookup(receiver.text, receiver, source)?.node
: undefined;
if (
declaration &&
(ts.isVariableDeclaration(declaration) || ts.isParameter(declaration)) &&
declaration.type
) {
const member = memberType(
declaration.type,
property.name,
env,
this.context,
this.resolver,
);
const signature = member && this.signature(member.type, member.env);
if (signature) {
return this.signatureTarget(signature);
}
}
if (declaration && ts.isNamespaceImport(declaration)) {
const specifier = (declaration.parent.parent.moduleSpecifier as ts.StringLiteral)
.text;
if (
specifier === packageName &&
(property.name === 'protobuf_encode' || property.name === 'protobuf_decode')
) {
return {
node: this.graph.runtimePlan(property.name).node,
bindings: new Map(),
};
}
const module = this.context.module(specifier, declaration.getSourceFile());
ref = module ? this.context.exported(property.name, module) : undefined;
} else {
const value = this.receivers.resolve(receiver, source, env, seen);
const member = value && this.receivers.member(value, property.name);
if (member) {
if (callable(member.node)) {
return { node: member.node, bindings: member.bindings };
}
if ('initializer' in member.node && member.node.initializer) {
const target = this.target(
member.node.initializer,
member.node.getSourceFile(),
member.env,
seen,
);
if (target) {
return {
node: target.node,
bindings: new Map([...member.bindings, ...target.bindings]),
};
}
}
if (ts.isShorthandPropertyAssignment(member.node)) {
return this.target(
member.node.name,
member.node.getSourceFile(),
member.env,
seen,
);
}
}
}
}
if (!ref) {
return;
}
// Ambient library declarations have no body in which to thread codecs.
// Their unrelated generic arguments must not enter schema inference.
if (ref.source.isDeclarationFile) {
return;
}
const runtime = runtimeImportName(ref.node);
if (runtime === 'protobuf_encode' || runtime === 'protobuf_decode') {
return { node: this.graph.runtimePlan(runtime).node, bindings: new Map() };
}
if ((ts.isParameter(ref.node) || ts.isVariableDeclaration(ref.node)) && ref.node.type) {
const signature = this.signature(ref.node.type, env);
if (signature) {
return this.signatureTarget(signature);
}
}
if (
ts.isBindingElement(ref.node) &&
ts.isObjectBindingPattern(ref.node.parent) &&
(ts.isVariableDeclaration(ref.node.parent.parent) ||
ts.isParameter(ref.node.parent.parent))
) {
const declaration = ref.node.parent.parent;
const initializer = declaration.initializer;
const name = ref.node.propertyName ?? ref.node.name;
if (declaration.type && (ts.isIdentifier(name) || ts.isStringLiteral(name))) {
const property = memberType(
declaration.type,
name.text,
env,
this.context,
this.resolver,
);
const signature = property && this.signature(property.type, property.env);
if (signature) {
return this.signatureTarget(signature);
}
}
if (initializer && (ts.isIdentifier(name) || ts.isStringLiteral(name))) {
return this.target(
ts.factory.createPropertyAccessExpression(initializer, name.text),
ref.source,
env,
seen,
);
}
}
if (callable(ref.node) || ts.isClassDeclaration(ref.node)) {
return { node: ref.node, bindings: new Map() };
}
if (ts.isVariableDeclaration(ref.node) && ref.node.initializer) {
if (callable(ref.node.initializer)) {
return { node: ref.node.initializer, bindings: new Map() };
}
return this.target(ref.node.initializer, ref.source, env, seen);
}
return;
}
specialize(
node: ts.CallExpression | ts.NewExpression,
target: Target,
env: TypeEnvironment,
): Target {
return {
node: target.node,
bindings: this.graph.bindings(
{
node,
target: this.graph.plan(target.node),
bindings: this.capture(node, target, env).bindings,
typeArgs: node.typeArguments,
env,
},
true,
),
};
}
capture(node: ts.Node, target: Target, env: TypeEnvironment): Target {
const bindings = new Map(target.bindings);
const lexical = this.graph.environment(node);
for (const [name, type] of this.graph.plan(target.node).env) {
if (
!bindings.has(type.key) &&
!target.node.typeParameters?.some((param) => param.name.text === name) &&
lexical.get(name)?.key === type.key &&
env.has(name)
) {
bindings.set(type.key, env.get(name)!);
}
}
return { node: target.node, bindings };
}
environment(target: Target): TypeEnvironment {
return new Map(
[...this.graph.plan(target.node).env].map(([name, type]) => [
name,
this.resolver.substitute(type, target.bindings),
]),
);
}
infer(
expression: ts.Expression,
source: ts.SourceFile,
seen = new Set<ts.Node>(),
): ts.TypeNode | undefined {
if (seen.has(expression)) {
return;
}
seen.add(expression);
if (
ts.isAsExpression(expression) ||
ts.isTypeAssertionExpression(expression) ||
ts.isSatisfiesExpression(expression)
) {
return expression.type;
}
if (ts.isIdentifier(expression)) {
const ref = this.context.lookup(expression.text, expression, source);
if (ref && (ts.isVariableDeclaration(ref.node) || ts.isParameter(ref.node))) {
if (ref.node.type) {
return ref.node.type;
}
if (ref.node.initializer) {
return this.infer(ref.node.initializer, ref.source, seen);
}
}
}
return;
}
}
+140
View File
@@ -0,0 +1,140 @@
import ts from 'typescript';
import { fail } from '../source/context.js';
import { callable } from './model.js';
import type { CallAnalysis } from './analysis.js';
export function validateSource(graph: CallAnalysis, source: ts.SourceFile): void {
if (![...graph.plans.values()].some((plan) => plan.requirements.size)) {
return;
}
const visit = (node: ts.Node): void => {
if (ts.isExpressionWithTypeArguments(node) && !ts.isHeritageClause(node.parent)) {
const target = graph.targets.target(node.expression, source, graph.environment(node));
if (
target &&
graph.plan(target.node).requirements.size &&
!graph.callbacks.adapters.has(node)
) {
fail(node, 'Generic function values require an explicitly typed arrow adapter.');
}
}
if (ts.isTypeNode(node) || ts.isImportDeclaration(node) || ts.isExportDeclaration(node)) {
return;
}
if (ts.isIdentifier(node) && node.text === 'arguments') {
const owner = graph.owner(node);
if (owner && graph.plan(owner).requirements.size) {
fail(
node,
'Generic codec functions cannot reflect on arguments; use named or rest parameters.',
);
}
}
if (
(ts.isIdentifier(node) ||
ts.isPropertyAccessExpression(node) ||
ts.isElementAccessExpression(node) ||
ts.isArrowFunction(node) ||
ts.isFunctionExpression(node)) &&
isValuePosition(node)
) {
const target = graph.targets.target(node, source, graph.environment(node));
const owner = target && graph.owner(target.node);
if (
target &&
!graph.callbacks.adapters.has(node) &&
!ts.isClassDeclaration(target.node) &&
(graph.plan(target.node).requirements.size ||
(ts.isMethodDeclaration(target.node) &&
owner &&
graph.plan(owner).requirements.size))
) {
fail(
node,
'Cannot infer callback schema; annotate the callback type or supply concrete type arguments.',
);
}
}
if (
(ts.isPropertyAccessExpression(node) || ts.isElementAccessExpression(node)) &&
ts.isVariableDeclaration(node.parent) &&
node.parent.initializer === node
) {
const target = graph.targets.target(node, source, graph.environment(node));
const owner = target && graph.owner(target.node);
if (
target &&
ts.isMethodDeclaration(target.node) &&
!target.node.typeParameters?.length &&
owner &&
ts.isClassDeclaration(owner) &&
graph.plan(owner).requirements.size
) {
fail(
node,
'Instance codec methods require their receiver; use an arrow adapter instead of detaching the method.',
);
}
}
ts.forEachChild(node, visit);
};
visit(source);
for (const site of graph.sites.get(source) ?? []) {
if (!site.direct?.type.dynamic || !site.owner || !ts.isClassDeclaration(site.owner.node)) {
continue;
}
for (
let parent = site.node.parent;
parent && parent !== site.owner.node;
parent = parent.parent
) {
if (callable(parent) || ts.isConstructorDeclaration(parent)) {
break;
}
if (ts.isPropertyDeclaration(parent)) {
fail(
site.node,
'Move generic codec calls from class field initializers into the constructor body.',
);
}
}
}
}
function isValuePosition(node: ts.Expression): boolean {
let value: ts.Node = node;
while (
ts.isParenthesizedExpression(value.parent) ||
ts.isAsExpression(value.parent) ||
ts.isSatisfiesExpression(value.parent)
) {
value = value.parent;
}
const parent = value.parent;
return (
(ts.isCallExpression(parent) && parent.expression !== value) ||
ts.isReturnStatement(parent) ||
ts.isShorthandPropertyAssignment(parent) ||
ts.isArrayLiteralExpression(parent) ||
(ts.isPropertyAssignment(parent) && parent.initializer === value) ||
(ts.isBinaryExpression(parent) &&
parent.right === value &&
parent.operatorToken.kind === ts.SyntaxKind.EqualsToken)
);
}
+74
View File
@@ -0,0 +1,74 @@
import type { Plugin, ResolvedConfig } from 'vite';
import { ProtobufCompiler } from './compiler/compiler.js';
import { buildEntries } from './source/entries.js';
import { packageName } from './package.js';
export default function protobufVitePlugin(): Plugin {
const compiler = new ProtobufCompiler();
const transformedFiles = new Set<string>();
let config: ResolvedConfig | undefined;
let needsPreparation = true;
const invalidate = () => {
compiler.clear();
needsPreparation = true;
};
return {
name: `${packageName}:vite`,
enforce: 'pre',
configResolved(resolved) {
config = resolved;
},
buildStart() {
invalidate();
transformedFiles.clear();
},
watchChange() {
invalidate();
},
transform(code, id) {
const cleanId = id.split('?')[0];
if (!/\.[cm]?tsx?$/.test(cleanId) || /\.d\.[cm]?ts$/.test(cleanId)) {
return null;
}
if (needsPreparation) {
if (config) {
compiler.prepare(buildEntries(config));
}
needsPreparation = false;
}
const transformed = compiler.compile(code, cleanId);
if (transformed !== null) {
transformedFiles.add(cleanId);
for (const dependency of compiler.context.files.keys()) {
this.addWatchFile?.(dependency);
}
}
return transformed === null ? null : { code: transformed, map: null };
},
handleHotUpdate({ file, server }) {
if (/\.[cm]?tsx?$/.test(file)) {
invalidate();
return [...transformedFiles].flatMap((id) => {
const module = server.moduleGraph.getModuleById(id);
return module ? [module] : [];
});
}
return;
},
};
}
+1
View File
@@ -0,0 +1 @@
export { name as packageName } from '../package.json';
+69
View File
@@ -0,0 +1,69 @@
import { packageName } from './package.js';
function uncompiled(operation: string): never {
throw new Error(
`${operation}<T>() was not transformed by the ${packageName} Vite plugin. Add the plugin from ${packageName}/vite to your build.`,
);
}
export function protobuf_encode<T>(_value: T): Uint8Array {
return uncompiled('protobuf_encode');
}
export function protobuf_decode<T>(_bytes: Uint8Array): T {
return uncompiled('protobuf_decode');
}
export interface ProtobufUnknownField {
fieldNumber: number;
wireType: number;
count: number;
totalByteLength: number;
}
export interface ProtobufUnknownFieldMetadata {
fields: readonly ProtobufUnknownField[];
totalOccurrences: number;
omittedOccurrences: number;
omittedByteLength: number;
}
const PROTOBUF_UNKNOWN_FIELDS = Symbol.for('snowluma.proton.unknownFields');
const EMPTY_UNKNOWN_FIELD_METADATA: ProtobufUnknownFieldMetadata = Object.freeze({
fields: Object.freeze([]),
totalOccurrences: 0,
omittedOccurrences: 0,
omittedByteLength: 0,
});
/** Unknown wire tags retained by generated decoders. The symbol-backed
* metadata is non-enumerable, so decoded object shapes and re-encoding remain
* unchanged while protocol boundaries can make schema drift observable. */
export function protobuf_getUnknownFields(value: unknown): readonly ProtobufUnknownField[] {
return protobuf_getUnknownFieldMetadata(value).fields;
}
export function protobuf_getUnknownFieldMetadata(value: unknown): ProtobufUnknownFieldMetadata {
if (typeof value !== 'object' || value === null) {
return EMPTY_UNKNOWN_FIELD_METADATA;
}
const metadata = (value as Record<PropertyKey, unknown>)[PROTOBUF_UNKNOWN_FIELDS];
if (typeof metadata !== 'object' || metadata === null) {
return EMPTY_UNKNOWN_FIELD_METADATA;
}
const candidate = metadata as Partial<ProtobufUnknownFieldMetadata>;
if (!Array.isArray(candidate.fields)) {
return EMPTY_UNKNOWN_FIELD_METADATA;
}
return {
fields: candidate.fields,
totalOccurrences: candidate.totalOccurrences ?? candidate.fields.length,
omittedOccurrences: candidate.omittedOccurrences ?? 0,
omittedByteLength: candidate.omittedByteLength ?? 0,
};
}
+143
View File
@@ -0,0 +1,143 @@
import type ts from 'typescript';
import { fail } from '../source/context.js';
import {
PRIMITIVE_TYPE_MAP,
WireType,
type MessageRegistry,
type ProtobufField,
} from '../codegen/model.js';
import type { TypeValue } from './types.js';
export class MessageBuilder {
readonly registry: MessageRegistry = new Map();
private readonly messageNames = new Map<string, string>();
private readonly nameOwners = new Map<string, string>();
build(type: TypeValue, node: ts.Node): string {
if (type.dynamic) {
fail(
node,
`Cannot statically resolve protobuf type ${type.name}. Supply concrete generic arguments at its call site.`,
);
}
if (type.kind !== 'object' || !type.fields) {
fail(node, `Expected a protobuf message, received ${type.name}.`);
}
const cached = this.messageNames.get(type.key);
if (cached) {
return cached;
}
let name = type.name.replace(/[^\p{ID_Continue}$]/gu, '_');
if (!/^[\p{ID_Start}_$]/u.test(name)) {
name = '_' + name;
}
if (this.nameOwners.has(name) && this.nameOwners.get(name) !== type.key) {
let hash = 2166136261;
for (const char of type.key) {
hash = Math.imul(hash ^ char.charCodeAt(0), 16777619);
}
name += '_' + (hash >>> 0).toString(36);
}
this.nameOwners.set(name, type.key);
this.messageNames.set(type.key, name);
const fields: ProtobufField[] = [];
this.registry.set(name, { name, fields });
const numbers = new Set<number>();
for (const [fieldName, property] of type.fields) {
let marker = property.type;
let optional = property.optional;
if (marker.kind === 'union') {
optional ||= marker.values!.some((item) => item.kind === 'null');
const members = marker.values!.filter((item) => item.kind !== 'null');
if (members.length !== 1) {
fail(property.node, `Ambiguous protobuf union on field ${fieldName}.`);
}
marker = members[0];
}
if (marker.kind !== 'marker') {
fail(
property.node,
`Field ${fieldName} must use pb<N, T>, pb_optional<N, T> or pb_repeated<N, T>.`,
);
}
const number = marker.fieldNumber!;
if (
!Number.isInteger(number) ||
number < 1 ||
number > 536870911 ||
(number >= 19000 && number <= 19999)
) {
fail(property.node, `Invalid or reserved protobuf field number ${number}.`);
}
if (numbers.has(number)) {
fail(property.node, `Duplicate protobuf field number ${number}.`);
}
numbers.add(number);
let payload = marker.values![0];
if (payload.kind === 'marker') {
payload = payload.values![0];
}
if (payload.kind === 'union') {
optional ||= payload.values!.some((item) => item.kind === 'null');
const members = payload.values!.filter((item) => item.kind !== 'null');
if (members.length !== 1) {
fail(property.node, `Ambiguous protobuf union on field ${fieldName}.`);
}
payload = members[0];
}
const repeated = marker.repeated || payload.kind === 'array';
if (payload.kind === 'array') {
payload = payload.values![0];
}
const isMessage = payload.kind === 'object';
const typeName = isMessage ? this.build(payload, property.node) : payload.name;
const primitive = PRIMITIVE_TYPE_MAP[typeName];
if (!isMessage && !primitive) {
fail(
property.node,
`Unsupported protobuf field type ${typeName}. Use an explicit protobuf scalar type.`,
);
}
fields.push({
name: fieldName,
fieldNumber: number,
typeName,
wireType: isMessage ? WireType.LengthDelim : primitive.wireType,
isMessage,
isRepeated: !!repeated,
isOptional: optional,
explicitPresence: marker.explicitPresence,
});
}
fields.sort((a, b) => a.fieldNumber - b.fieldNumber);
return name;
}
}
+249
View File
@@ -0,0 +1,249 @@
import ts from 'typescript';
import { fail, type DeclarationRef } from '../source/context.js';
import type { SchemaResolver } from './resolver.js';
import { value, type TypeEnvironment, type TypeValue } from './types.js';
export class ObjectTypes {
private readonly types = new Map<string, TypeValue>();
private readonly aliases = new Set<string>();
private readonly concrete = new WeakMap<ts.Node, TypeValue>();
private readonly bases = new Set<string>();
constructor(private readonly resolver: SchemaResolver) {}
declaration(
ref: DeclarationRef,
args: readonly ts.TypeNode[],
caller: ts.SourceFile,
env: TypeEnvironment,
origin: ts.TypeNode,
heritage = false,
): TypeValue {
const node = ref.node;
if (
!ts.isInterfaceDeclaration(node) &&
!ts.isTypeAliasDeclaration(node) &&
!ts.isClassDeclaration(node)
) {
fail(origin, 'Expected an interface, type alias or class schema.');
}
const concrete = !node.typeParameters?.length && ts.isSourceFile(node.parent);
const previous = concrete && this.concrete.get(node);
if (previous && heritage && this.bases.has(previous.key)) {
fail(origin, 'Circular schema inheritance.');
}
if (previous) {
return previous;
}
const local = this.resolver.arguments(node.typeParameters, args, caller, env, node);
for (let scope = node.parent; scope; scope = scope.parent) {
if (!ts.isFunctionLike(scope) && !ts.isClassDeclaration(scope)) {
continue;
}
for (const parameter of scope.typeParameters ?? []) {
const name = parameter.name.text;
if (!local.has(name)) {
local.set(name, env.get(name) ?? this.resolver.parameter(name, scope));
}
}
}
const parameters = [...local.values()];
const key = `${ref.source.fileName}:${node.pos}<${parameters.map((type) => type.key).join(',')}>`;
if (heritage && this.bases.has(key)) {
fail(origin, 'Circular schema inheritance.');
}
if (heritage && this.bases.size >= 128) {
fail(origin, 'Schema inheritance exceeds 128 levels.');
}
const cached = this.types.get(key);
if (cached) {
return cached;
}
if (ts.isTypeAliasDeclaration(node)) {
if (ts.isTypeLiteralNode(node.type)) {
const result = this.object(
node.name.text + parameters.map((type) => '__' + type.name).join(''),
key,
);
result.origin = { node: origin, source: caller, env: new Map(env) };
if (parameters.length) {
result.application = { declaration: node, arguments: parameters };
}
this.types.set(key, result);
if (concrete) {
this.concrete.set(node, result);
}
return this.members(node.type.members, result, ref.source, local);
}
if (this.aliases.has(key)) {
fail(node, 'Circular type alias.');
}
this.aliases.add(key);
try {
const result = this.resolver.type(node.type, ref.source, local);
this.types.set(key, result);
if (concrete) {
this.concrete.set(node, result);
}
return result;
} finally {
this.aliases.delete(key);
}
}
const name =
(node.name?.text ?? 'Default') + parameters.map((type) => '__' + type.name).join('');
const result = this.object(name, key);
result.dynamic = parameters.some((type) => type.dynamic);
if (parameters.length) {
result.application = { declaration: node, arguments: parameters };
}
result.origin = { node: origin, source: caller, env: new Map(env) };
this.types.set(key, result);
if (concrete) {
this.concrete.set(node, result);
}
this.bases.add(key);
try {
for (const clause of node.heritageClauses ?? []) {
if (clause.token !== ts.SyntaxKind.ExtendsKeyword) {
continue;
}
for (const base of clause.types) {
const declaration = this.resolver.context.reference(
base.expression,
ref.source,
true,
);
if (!declaration) {
fail(base, 'Unresolved base schema.');
}
this.merge(
result,
this.declaration(
declaration,
base.typeArguments ?? [],
ref.source,
local,
origin,
true,
),
base,
);
}
}
} finally {
this.bases.delete(key);
}
return this.members(node.members, result, ref.source, local);
}
members(
members: ts.NodeArray<ts.TypeElement | ts.ClassElement>,
result: TypeValue,
source: ts.SourceFile,
env: TypeEnvironment,
): TypeValue {
const fields = members.flatMap<ts.TypeElement | ts.ClassElement | ts.ParameterDeclaration>(
(member) =>
ts.isConstructorDeclaration(member)
? member.parameters.filter((parameter) =>
parameter.modifiers?.some((modifier) =>
[
ts.SyntaxKind.PublicKeyword,
ts.SyntaxKind.PrivateKeyword,
ts.SyntaxKind.ProtectedKeyword,
ts.SyntaxKind.ReadonlyKeyword,
].includes(modifier.kind),
),
)
: [member],
);
for (const member of fields) {
if (
!ts.isPropertySignature(member) &&
!ts.isPropertyDeclaration(member) &&
!ts.isParameter(member)
) {
continue;
}
if (
member.modifiers?.some((modifier) => modifier.kind === ts.SyntaxKind.StaticKeyword)
) {
continue;
}
if (!member.type || !member.name || ts.isComputedPropertyName(member.name)) {
fail(member, 'Schema fields require static names and explicit field markers.');
}
const name =
ts.isIdentifier(member.name) ||
ts.isStringLiteral(member.name) ||
ts.isNumericLiteral(member.name)
? member.name.text
: fail(member, 'Unsupported field name.');
const type = this.resolver.type(member.type, source, env);
result.fields!.set(name, { type, optional: !!member.questionToken, node: member });
result.dynamic ||= type.dynamic;
}
return result;
}
merge(target: TypeValue, source: TypeValue, node: ts.Node): void {
if (!source.fields) {
fail(node, 'Schema intersections and bases must be object types.');
}
for (const [name, property] of source.fields) {
const existing = target.fields!.get(name);
if (existing && existing.type.key !== property.type.key) {
fail(node, `Conflicting inherited field ${name}.`);
}
target.fields!.set(name, property);
}
target.dynamic ||= source.dynamic;
}
object(name: string, key: string): TypeValue {
return { ...value('object', name, key), fields: new Map() };
}
}
+246
View File
@@ -0,0 +1,246 @@
import ts from 'typescript';
import { fail } from '../source/context.js';
import type { SchemaResolver } from './resolver.js';
import { assignable } from './relations.js';
import { value, type TypeEnvironment, type TypeValue } from './types.js';
const utilities = new Set([
'Promise',
'Awaited',
'ReturnType',
'Parameters',
'InstanceType',
'ConstructorParameters',
'Extract',
'Exclude',
'Uppercase',
'Lowercase',
'Capitalize',
'Uncapitalize',
]);
function container(kind: TypeValue['kind'], values: TypeValue[]): TypeValue {
return {
kind,
values,
key: `${kind}<${values.map((type) => type.key)}>`,
name: kind,
dynamic: values.some((type) => type.dynamic),
};
}
export function resolveOperator(
node: ts.TypeNode,
source: ts.SourceFile,
env: TypeEnvironment,
resolver: SchemaResolver,
): TypeValue | undefined {
if (
ts.isTypeReferenceNode(node) &&
(!ts.isIdentifier(node.typeName) || !utilities.has(node.typeName.text))
) {
return;
}
const resolve = (type: ts.TypeNode) => resolver.type(type, type.getSourceFile(), env);
if (ts.isInferTypeNode(node)) {
return value('inference', node.typeParameter.name.text);
}
if (ts.isTupleTypeNode(node)) {
return container(
'tuple',
node.elements.map((type) => resolve(ts.isNamedTupleMember(type) ? type.type : type)),
);
}
if (ts.isOptionalTypeNode(node)) {
return container('union', [resolve(node.type), value('null', 'undefined')]);
}
if (ts.isRestTypeNode(node)) {
return resolve(node.type);
}
if (ts.isFunctionTypeNode(node) || ts.isConstructorTypeNode(node)) {
const local = new Map(env);
for (const param of node.typeParameters ?? []) {
local.set(param.name.text, resolver.parameter(param.name.text, node));
}
const parameters = node.parameters.filter(
(param) => !ts.isIdentifier(param.name) || param.name.text !== 'this',
);
return container(ts.isFunctionTypeNode(node) ? 'function' : 'constructor', [
container(
'tuple',
parameters.map((param) =>
param.type
? resolver.type(param.type, source, local)
: value('primitive', 'unknown'),
),
),
resolver.type(node.type, source, local),
]);
}
if (ts.isTypeQueryNode(node)) {
const ref = ts.isIdentifier(node.exprName)
? resolver.context.lookup(node.exprName.text, node.exprName, source)
: resolver.context.typeName(node.exprName, source);
const declaration = ref?.node;
if (declaration && (ts.isVariableDeclaration(declaration) || ts.isParameter(declaration))) {
if (declaration.type) {
return resolve(declaration.type);
}
const initializer = declaration.initializer;
if (
initializer &&
(ts.isAsExpression(initializer) || ts.isSatisfiesExpression(initializer))
) {
return resolve(initializer.type);
}
}
if (declaration && ts.isFunctionDeclaration(declaration) && declaration.type) {
return container('function', [
container(
'tuple',
declaration.parameters.map((param) =>
param.type ? resolve(param.type) : value('primitive', 'unknown'),
),
),
resolve(declaration.type),
]);
}
fail(
node,
'Type queries used in schemas require an explicit variable or function return annotation.',
);
}
if (ts.isTemplateLiteralTypeNode(node)) {
let strings = [node.head.text];
for (const span of node.templateSpans) {
const type = resolve(span.type);
if (type.dynamic) {
return resolver.defer(node, source, env);
}
const options = type.kind === 'union' ? type.values! : [type];
if (options.some((type) => type.kind !== 'literal')) {
fail(node, 'Template schema keys require literal types.');
}
strings = strings.flatMap((prefix) =>
options.map((type) => prefix + type.name + span.literal.text),
);
if (strings.length > 128) {
fail(node, 'Template type expansion exceeds 128 keys.');
}
}
const literals = strings.map((text) => value('literal', text, JSON.stringify(text)));
return literals.length === 1 ? literals[0] : container('union', literals);
}
if (!ts.isTypeReferenceNode(node) || !ts.isIdentifier(node.typeName)) {
return;
}
const name = node.typeName.text;
if (!utilities.has(name) || resolver.context.typeName(node.typeName, source)) {
return;
}
const args = node.typeArguments;
if (!args?.length) {
fail(node, `${name} requires a type argument.`);
}
const operand = resolve(args[0]);
if (name === 'Promise') {
return container('promise', [operand]);
}
if (operand.dynamic) {
return resolver.defer(node, source, env);
}
if (name === 'Awaited') {
const unwrap = (type: TypeValue): TypeValue =>
type.kind === 'promise'
? unwrap(type.values![0])
: type.kind === 'union'
? container('union', type.values!.map(unwrap))
: type;
return unwrap(operand);
}
if (['ReturnType', 'Parameters', 'InstanceType', 'ConstructorParameters'].includes(name)) {
const expected =
name === 'InstanceType' || name === 'ConstructorParameters'
? 'constructor'
: 'function';
if (operand.kind !== expected) {
fail(node, `${name} requires a ${expected} type.`);
}
return operand.values![name.endsWith('Parameters') ? 0 : 1];
}
if (name === 'Extract' || name === 'Exclude') {
if (!args[1]) {
fail(node, `${name} requires two type arguments.`);
}
const constraint = resolve(args[1]);
if (constraint.dynamic) {
return resolver.defer(node, source, env);
}
const selected = (operand.kind === 'union' ? operand.values! : [operand]).filter(
(type) => assignable(type, constraint) === (name === 'Extract'),
);
return selected.length === 0
? value('primitive', 'never')
: selected.length === 1
? selected[0]
: container('union', selected);
}
const transform = (type: TypeValue): TypeValue => {
if (type.kind !== 'literal' || !/^["']/.test(type.key)) {
fail(node, `${name} requires literal string keys.`);
}
const text =
name === 'Uppercase'
? type.name.toUpperCase()
: name === 'Lowercase'
? type.name.toLowerCase()
: (name === 'Capitalize'
? (type.name[0] ?? '').toUpperCase()
: (type.name[0] ?? '').toLowerCase()) + type.name.slice(1);
return value('literal', text, JSON.stringify(text));
};
return operand.kind === 'union'
? container('union', operand.values!.map(transform))
: transform(operand);
}
+137
View File
@@ -0,0 +1,137 @@
import type { TypeValue } from './types.js';
export function assignable(
source: TypeValue,
target: TypeValue,
seen = new Set<string>(),
): boolean {
if (
source.key === target.key ||
target.name === 'unknown' ||
target.name === 'any' ||
source.name === 'never'
) {
return true;
}
if (target.name === 'object') {
return (
!!source.fields ||
['array', 'tuple', 'function', 'constructor', 'promise'].includes(source.kind)
);
}
const key = `${source.key}:${target.key}`;
if (seen.has(key)) {
return true;
}
seen.add(key);
if (source.kind === 'marker') {
return assignable(source.values![0], target, seen);
}
if (target.kind === 'marker') {
return assignable(source, target.values![0], seen);
}
if (source.kind === 'union') {
return source.values!.every((type) => assignable(type, target, new Set(seen)));
}
if (target.kind === 'union') {
return target.values!.some((type) => assignable(source, type, new Set(seen)));
}
if (source.kind === target.kind && source.values && target.values) {
return (
source.values.length === target.values.length &&
source.values.every((type, index) => assignable(type, target.values![index], seen))
);
}
if (source.fields && target.fields) {
return [...target.fields].every(([name, field]) => {
const actual = source.fields!.get(name);
return actual
? (field.optional || !actual.optional) && assignable(actual.type, field.type, seen)
: field.optional;
});
}
if (target.kind === 'primitive' && source.kind === 'literal') {
return target.name === 'string'
? /^["']/.test(source.key)
: target.name === 'number'
? !Number.isNaN(Number(source.name))
: target.name === 'bool' && /^(true|false)$/.test(source.name);
}
if (target.name === 'number') {
return /^(?:u?int_32|sint_32|s?fixed_32|float|double)$/.test(source.name);
}
if (target.name === 'bigint') {
return /^(?:u?int_64|sint_64|s?fixed_64)$/.test(source.name);
}
return false;
}
export function matchInferred(
source: TypeValue,
pattern: TypeValue,
inferred: Map<string, TypeValue>,
seen = new Set<string>(),
): boolean {
if (pattern.kind === 'inference') {
const previous = inferred.get(pattern.name);
if (previous && previous.key !== source.key) {
return false;
}
inferred.set(pattern.name, source);
return true;
}
const key = `${source.key}:${pattern.key}`;
if (seen.has(key)) {
return true;
}
seen.add(key);
if (source.kind === 'function' && pattern.kind === 'function') {
const args = pattern.values![0].values!;
const variadicAny =
args.length === 1 && args[0].kind === 'array' && args[0].values![0].name === 'any';
return (
(variadicAny || matchInferred(source.values![0], pattern.values![0], inferred, seen)) &&
matchInferred(source.values![1], pattern.values![1], inferred, seen)
);
}
if (source.fields && pattern.fields) {
return [...pattern.fields].every(([name, property]) => {
const actual = source.fields!.get(name);
return actual
? matchInferred(actual.type, property.type, inferred, seen)
: property.optional;
});
}
if (source.kind === pattern.kind && source.values && pattern.values) {
return (
source.values.length === pattern.values.length &&
source.values.every((type, index) =>
matchInferred(type, pattern.values![index], inferred, seen),
)
);
}
return assignable(source, pattern);
}
+600
View File
@@ -0,0 +1,600 @@
import { packageName } from '../package.js';
import ts from 'typescript';
import { SourceContext, fail, runtimeImportName } from '../source/context.js';
import { PRIMITIVE_TYPE_MAP } from '../codegen/model.js';
import { value, type TypeEnvironment, type TypeValue } from './types.js';
import { MessageBuilder } from './message.js';
import { matchInferred } from './relations.js';
import { resolveOperator } from './operators.js';
import { ObjectTypes } from './objects.js';
const primitiveNames = new Set(Object.keys(PRIMITIVE_TYPE_MAP));
/** Lazily resolves only types reached by codec calls; no project-wide type checker. */
export class SchemaResolver {
private readonly messages = new MessageBuilder();
readonly registry = this.messages.registry;
private readonly objects = new ObjectTypes(this);
private depth = 0;
private readonly resolvedLazy = new WeakMap<TypeValue, TypeValue>();
constructor(readonly context: SourceContext) {}
parameter(name: string, owner: ts.Node): TypeValue {
return value('parameter', name, `${owner.getSourceFile().fileName}:${owner.pos}:${name}`);
}
substitute(type: TypeValue, bindings: Map<string, TypeValue>): TypeValue {
if (type.kind === 'lazy') {
const { node, source, env } = type.origin!;
return this.lazy(
node,
source,
new Map([...env].map(([name, val]) => [name, this.substitute(val, bindings)])),
);
}
if (!type.dynamic) {
return type;
}
if (type.kind === 'parameter') {
return bindings.get(type.key) ?? type;
}
if (type.origin) {
const { node, source, env } = type.origin;
return this.type(
node,
source,
new Map([...env].map(([name, val]) => [name, this.substitute(val, bindings)])),
);
}
return type;
}
type(
node: ts.TypeNode,
source = node.getSourceFile(),
env: TypeEnvironment = new Map(),
): TypeValue {
if (++this.depth > 128) {
this.depth--;
return fail(
node,
'Type expansion exceeds 128 levels (possibly an expanding recursive generic).',
);
}
try {
const result = this.resolve(node, source, env);
if (result.dynamic && result.kind !== 'parameter' && !result.origin) {
result.origin = { node, source, env: new Map(env) };
}
return result;
} finally {
this.depth--;
}
}
arguments(
params: readonly ts.TypeParameterDeclaration[] | undefined,
args: readonly ts.TypeNode[] | undefined,
source: ts.SourceFile,
outer: TypeEnvironment,
declaration: ts.Node,
lazy = false,
): TypeEnvironment {
const result = new Map<string, TypeValue>();
if ((args?.length ?? 0) > (params?.length ?? 0)) {
fail(declaration, 'Too many generic type arguments.');
}
for (let i = 0; i < (params?.length ?? 0); i++) {
const param = params![i];
const arg = args?.[i];
const resolved = arg
? lazy
? this.lazy(arg, source, outer)
: this.type(arg, source, outer)
: param.default
? lazy
? this.lazy(param.default, declaration.getSourceFile(), result)
: this.type(param.default, declaration.getSourceFile(), result)
: undefined;
if (!resolved) {
fail(declaration, `Missing concrete type argument for ${param.name.text}.`);
}
result.set(param.name.text, resolved);
}
return result;
}
message(type: TypeValue, node: ts.Node): string {
return this.messages.build(this.materialize(type), node);
}
/** Receiver discovery carries application types without expanding them until
* a protobuf requirement actually consumes the binding. */
lazy(node: ts.TypeNode, source: ts.SourceFile, env: TypeEnvironment): TypeValue {
if (ts.isTypeReferenceNode(node) && ts.isIdentifier(node.typeName)) {
const bound = env.get(node.typeName.text);
if (bound) {
return bound;
}
if (primitiveNames.has(node.typeName.text)) {
return this.type(node, source, env);
}
}
const mentionsParameter = (child: ts.Node): boolean =>
(ts.isIdentifier(child) && env.get(child.text)?.dynamic === true) ||
(ts.forEachChild(child, mentionsParameter) ?? false);
return {
...value(
'lazy',
node.getText(source),
`lazy:${source.fileName}:${node.getText(source)}<${[...env.values()].map((v) => v.key)}>`,
),
dynamic: mentionsParameter(node),
origin: { node, source, env: new Map(env) },
};
}
private materialize(type: TypeValue): TypeValue {
if (type.kind !== 'lazy') {
return type;
}
const cached = this.resolvedLazy.get(type);
if (cached) {
return cached;
}
const { node, source, env } = type.origin!;
const resolved = this.type(node, source, env);
this.resolvedLazy.set(type, resolved);
return resolved;
}
private resolve(node: ts.TypeNode, source: ts.SourceFile, env: TypeEnvironment): TypeValue {
if (ts.isTypeReferenceNode(node)) {
return this.reference(node, source, env);
}
const operator = resolveOperator(node, source, env, this);
if (operator) {
return operator;
}
if (ts.isParenthesizedTypeNode(node)) {
return this.type(node.type, source, env);
}
if (node.kind === ts.SyntaxKind.StringKeyword) {
return value('primitive', 'string');
}
if (node.kind === ts.SyntaxKind.BooleanKeyword) {
return value('primitive', 'bool');
}
if (
[
ts.SyntaxKind.ObjectKeyword,
ts.SyntaxKind.NumberKeyword,
ts.SyntaxKind.BigIntKeyword,
ts.SyntaxKind.AnyKeyword,
ts.SyntaxKind.UnknownKeyword,
ts.SyntaxKind.NeverKeyword,
].includes(node.kind)
) {
return value('primitive', node.getText(source));
}
if (
node.kind === ts.SyntaxKind.UndefinedKeyword ||
node.kind === ts.SyntaxKind.VoidKeyword
) {
return value('null', 'undefined');
}
if (ts.isLiteralTypeNode(node)) {
if (node.literal.kind === ts.SyntaxKind.NullKeyword) {
return value('null', 'null');
}
if (ts.isStringLiteral(node.literal)) {
return value('literal', node.literal.text, JSON.stringify(node.literal.text));
}
return value('literal', node.literal.getText(source));
}
if (ts.isArrayTypeNode(node)) {
return this.container(
'array',
[this.type(node.elementType, source, env)],
node,
source,
env,
);
}
if (ts.isUnionTypeNode(node)) {
const members = [
...new Map(
node.types
.flatMap((item) => {
const type = this.type(item, source, env);
return type.kind === 'union' ? type.values! : [type];
})
.map((item) => [item.key, item]),
).values(),
];
return members.length === 1
? members[0]
: this.container('union', members, node, source, env);
}
if (ts.isIntersectionTypeNode(node)) {
const members = node.types.map((item) => this.type(item, source, env));
if (members.some((item) => item.dynamic && item.kind !== 'object')) {
return this.defer(node, source, env);
}
const result = this.objects.object(
'Intersection',
members.map((item) => item.key).join('&'),
);
for (const member of members) {
this.objects.merge(result, member, node);
}
return result;
}
if (ts.isTypeLiteralNode(node)) {
return this.objects.members(
node.members,
this.objects.object(
'Anonymous',
`${source.fileName}:${node.pos}<${[...env.values()].map((type) => type.key)}>`,
),
source,
env,
);
}
if (ts.isTypeOperatorNode(node)) {
const operand = this.type(node.type, source, env);
if (node.operator === ts.SyntaxKind.ReadonlyKeyword) {
return operand;
}
if (operand.dynamic && !operand.fields) {
return this.defer(node, source, env);
}
if (node.operator === ts.SyntaxKind.KeyOfKeyword && operand.fields) {
return this.container(
'union',
[...operand.fields.keys()].map((key) =>
value('literal', key, JSON.stringify(key)),
),
node,
source,
env,
);
}
}
if (ts.isIndexedAccessTypeNode(node)) {
const object = this.type(node.objectType, source, env);
const index = this.type(node.indexType, source, env);
if (object.dynamic || index.dynamic) {
return this.defer(node, source, env);
}
const keys = index.kind === 'union' ? index.values! : [index];
const values = keys.map(
(key) =>
object.fields?.get(key.name)?.type ??
(object.kind === 'tuple'
? object.values?.[Number(key.name)]
: object.kind === 'array'
? object.values?.[0]
: undefined) ??
fail(node, `Unknown indexed property ${key.name}.`),
);
return values.length === 1
? values[0]
: this.container('union', values, node, source, env);
}
if (ts.isConditionalTypeNode(node)) {
const check = this.type(node.checkType, source, env);
const constraint = this.type(node.extendsType, source, env);
if (check.dynamic || constraint.dynamic) {
return this.defer(node, source, env);
}
if (
check.kind === 'union' &&
ts.isTypeReferenceNode(node.checkType) &&
ts.isIdentifier(node.checkType.typeName) &&
env.has(node.checkType.typeName.text)
) {
const results = check.values!.map((type) =>
this.type(
node,
source,
new Map(env).set(
(node.checkType as ts.TypeReferenceNode).typeName.getText(source),
type,
),
),
);
const distinct = [...new Map(results.map((type) => [type.key, type])).values()];
return distinct.length === 1
? distinct[0]
: this.container('union', distinct, node, source, env);
}
const inferred = new Map<string, TypeValue>();
const matches = matchInferred(check, constraint, inferred);
return this.type(
matches ? node.trueType : node.falseType,
source,
new Map([...env, ...inferred]),
);
}
if (ts.isMappedTypeNode(node) && node.type && node.typeParameter.constraint) {
const keys = this.type(node.typeParameter.constraint, source, env);
if (keys.dynamic) {
return this.defer(node, source, env);
}
const result = this.objects.object(
'Mapped',
`${source.fileName}:${node.pos}<${[...env.values()].map((v) => v.key)}>`,
);
for (const key of keys.kind === 'union' ? keys.values! : [keys]) {
const local = new Map(env).set(node.typeParameter.name.text, key);
const remapped = node.nameType ? this.type(node.nameType, source, local) : key;
if (remapped.name === 'never') {
continue;
}
const inherited = ts.isIndexedAccessTypeNode(node.type)
? (this.type(node.type.objectType, source, local).fields?.get(key.name)
?.optional ?? false)
: false;
const optional = node.questionToken
? node.questionToken.kind !== ts.SyntaxKind.MinusToken
: inherited;
for (const name of remapped.kind === 'union' ? remapped.values! : [remapped]) {
result.fields!.set(name.name, {
type: this.type(node.type, source, local),
optional,
node,
});
}
}
return result;
}
fail(node, `Unsupported or ambiguous type expression ${node.getText(source)}.`);
}
private reference(
node: ts.TypeReferenceNode,
source: ts.SourceFile,
env: TypeEnvironment,
): TypeValue {
const simple = ts.isIdentifier(node.typeName) ? node.typeName.text : undefined;
if (simple && env.has(simple)) {
return this.materialize(env.get(simple)!);
}
const operator = resolveOperator(node, source, env, this);
if (operator) {
return operator;
}
const ref = this.context.typeName(node.typeName, source);
let name = simple
? (ref && runtimeImportName(ref.node)) || simple
: node.typeName.getText(source);
if (ts.isQualifiedName(node.typeName) && ts.isIdentifier(node.typeName.left)) {
const namespace = this.context.lookup(node.typeName.left.text, node, source, true);
if (
namespace &&
ts.isNamespaceImport(namespace.node) &&
(namespace.node.parent.parent.moduleSpecifier as ts.StringLiteral).text ===
packageName
) {
name = node.typeName.right.text;
}
}
const args = node.typeArguments ?? [];
if (
ref &&
(ts.isImportSpecifier(ref.node) || ts.isImportClause(ref.node)) &&
!runtimeImportName(ref.node)
) {
fail(node, `Unresolved imported type ${name}.`);
}
if (
ref &&
!runtimeImportName(ref.node) &&
!ts.isImportSpecifier(ref.node) &&
!ts.isNamespaceImport(ref.node)
) {
return this.objects.declaration(ref, args, source, env, node);
}
if (primitiveNames.has(name)) {
return value('primitive', name);
}
if (name === 'pb' || name === 'pb_repeated' || name === 'pb_optional') {
if (args.length !== 2) {
fail(node, `${name} requires a field number and a field type.`);
}
const fieldNumber = this.type(args[0], source, env);
const payload = this.type(args[1], source, env);
const marker = this.container('marker', [payload], node, source, env);
return {
...marker,
key: `${name}:${marker.key}:${fieldNumber.key}`,
dynamic: fieldNumber.dynamic || marker.dynamic,
fieldNumber: Number(fieldNumber.name),
repeated: name === 'pb_repeated',
explicitPresence: name === 'pb_optional',
};
}
if (name === 'Array' || name === 'ReadonlyArray') {
return this.container('array', [this.type(args[0], source, env)], node, source, env);
}
if (['Partial', 'Required', 'Readonly', 'Pick', 'Omit', 'NonNullable'].includes(name)) {
const operand = this.type(args[0], source, env);
if (operand.dynamic) {
return this.defer(node, source, env);
}
if (name === 'Readonly') {
return operand;
}
if (name === 'NonNullable') {
const members =
operand.kind === 'union'
? operand.values!.filter((item) => item.kind !== 'null')
: [operand];
return members.length === 1
? members[0]
: this.container('union', members, node, source, env);
}
if (!operand.fields) {
fail(node, `${name} requires an object type.`);
}
const keys = args[1] ? this.type(args[1], source, env) : undefined;
const selected = new Set(
keys ? (keys.kind === 'union' ? keys.values! : [keys]).map((key) => key.name) : [],
);
if (name === 'Pick') {
for (const key of selected) {
if (!operand.fields.has(key)) {
fail(node, `Unknown Pick property ${key}.`);
}
}
}
const result = this.objects.object(
`${name}__${operand.name}`,
`${name}<${operand.key},${keys?.key ?? ''}>`,
);
for (const [key, property] of operand.fields) {
if (
(name === 'Pick' && !selected.has(key)) ||
(name === 'Omit' && selected.has(key))
) {
continue;
}
result.fields!.set(key, {
...property,
optional: name === 'Partial' || (name !== 'Required' && property.optional),
});
}
return result;
}
if (ref) {
return this.objects.declaration(ref, args, source, env, node);
}
fail(node, `Unresolved type ${name}.`);
}
defer(node: ts.TypeNode, source: ts.SourceFile, env: TypeEnvironment): TypeValue {
return {
...value(
'deferred',
node.getText(source),
`${source.fileName}:${node.pos}<${[...env.values()].map((v) => v.key)}>`,
),
dynamic: true,
origin: { node, source, env: new Map(env) },
};
}
private container(
kind: TypeValue['kind'],
values: TypeValue[],
node: ts.TypeNode,
source: ts.SourceFile,
env: TypeEnvironment,
): TypeValue {
if (kind === 'union') {
values = values.filter((type) => type.name !== 'never');
if (!values.length) {
return value('primitive', 'never');
}
if (values.length === 1) {
return values[0];
}
}
return {
kind,
name: values.map((type) => type.name).join('__'),
key: `${kind}<${JSON.stringify(values.map((type) => type.key))}>`,
values,
dynamic: values.some((type) => type.dynamic),
origin: { node, source, env: new Map(env) },
};
}
}
+38
View File
@@ -0,0 +1,38 @@
import type ts from 'typescript';
export interface TypeValue {
kind:
| 'primitive'
| 'literal'
| 'null'
| 'parameter'
| 'object'
| 'marker'
| 'union'
| 'array'
| 'tuple'
| 'function'
| 'constructor'
| 'promise'
| 'inference'
| 'lazy'
| 'deferred';
key: string;
name: string;
dynamic: boolean;
fields?: Map<string, { type: TypeValue; optional: boolean; node: ts.Node }>;
values?: TypeValue[];
fieldNumber?: number;
repeated?: boolean;
explicitPresence?: boolean;
origin?: { node: ts.TypeNode; source: ts.SourceFile; env: TypeEnvironment };
application?: { declaration: ts.Declaration; arguments: TypeValue[] };
}
export type TypeEnvironment = Map<string, TypeValue>;
export const value = (kind: TypeValue['kind'], name: string, key = name): TypeValue => ({
kind,
key,
name,
dynamic: kind === 'parameter',
});
+423
View File
@@ -0,0 +1,423 @@
import { packageName } from '../package.js';
import ts from 'typescript';
import { readFileSync, statSync } from 'node:fs';
import { dirname, resolve } from 'node:path';
export interface DeclarationRef {
node: ts.Declaration;
source: ts.SourceFile;
}
export function runtimeImportName(node: ts.Declaration): string | undefined {
const declaration = ts.isImportSpecifier(node)
? node.parent.parent.parent
: ts.isExportSpecifier(node)
? node.parent.parent
: undefined;
if (
declaration?.moduleSpecifier &&
ts.isStringLiteral(declaration.moduleSpecifier) &&
declaration.moduleSpecifier.text === packageName
) {
return (
(node as ts.ImportSpecifier | ts.ExportSpecifier).propertyName?.text ??
(node as ts.ImportSpecifier | ts.ExportSpecifier).name.text
);
}
return;
}
const extensions = ['.ts', '.tsx', '.mts', '.cts', '/index.ts', '/index.tsx'];
export const normalizePath = (path: string): string => resolve(path).replace(/\\/g, '/');
/** One parse per source revision, shared by schema and generic-call analysis. */
export class SourceContext {
readonly files = new Map<string, ts.SourceFile>();
revision = 0;
private readonly modules = new Map<string, string | null>();
private readonly names = new WeakMap<ts.Node, Map<string, ts.Declaration>>();
private readonly typeNames = new WeakMap<ts.Node, Map<string, ts.Declaration>>();
private readonly options = new Map<string, ts.CompilerOptions>();
clear(): void {
this.files.clear();
this.modules.clear();
this.options.clear();
}
source(path: string, code?: string): ts.SourceFile {
const key = normalizePath(path);
const cached = this.files.get(key);
if (cached && (code === undefined || cached.text === code)) {
return cached;
}
if (cached) {
this.revision++;
}
const source = ts.createSourceFile(
key,
code ?? readFileSync(key, 'utf8'),
ts.ScriptTarget.Latest,
true,
);
this.files.set(key, source);
return source;
}
module(specifier: string, source: ts.SourceFile): ts.SourceFile | undefined {
if (specifier === packageName) {
return;
}
const key = `${source.fileName}:${specifier}`;
let path = this.modules.get(key);
if (path === undefined) {
if (specifier.startsWith('.')) {
const base = normalizePath(resolve(dirname(source.fileName), specifier));
const stem = base.replace(/\.(?:mjs|cjs|js)$/, '');
path =
[base, ...extensions.map((ext) => stem + ext)].find((candidate) => {
try {
return statSync(candidate).isFile();
} catch {
return false;
}
}) ?? null;
} else {
const directory = dirname(source.fileName);
let options = this.options.get(directory);
if (!options) {
const config = ts.findConfigFile(directory, ts.sys.fileExists);
options = config ? this.options.get(config) : undefined;
options ??= config
? ts.parseJsonConfigFileContent(
ts.readConfigFile(config, ts.sys.readFile).config,
{ ...ts.sys, readDirectory: () => [] },
dirname(config),
).options
: {};
if (config) {
this.options.set(config, options);
}
this.options.set(directory, options);
}
path =
ts.resolveModuleName(specifier, source.fileName, options, ts.sys).resolvedModule
?.resolvedFileName ?? null;
}
this.modules.set(key, path);
}
return path ? this.source(path) : undefined;
}
lookup(
name: string,
location: ts.Node,
source = location.getSourceFile(),
typeOnly = false,
): DeclarationRef | undefined {
for (let scope: ts.Node | undefined = location; scope; scope = scope.parent) {
if (!typeOnly && ts.isFunctionLike(scope)) {
const parameter = scope.parameters.find(
(param) => ts.isIdentifier(param.name) && param.name.text === name,
);
if (parameter) {
return { node: parameter, source };
}
for (const param of scope.parameters) {
if (!ts.isObjectBindingPattern(param.name)) {
continue;
}
const element = param.name.elements.find(
(element) => ts.isIdentifier(element.name) && element.name.text === name,
);
if (element) {
return { node: element, source };
}
}
}
if (!ts.isSourceFile(scope) && !ts.isBlock(scope) && !ts.isModuleBlock(scope)) {
continue;
}
const declaration = this.declarations(scope, typeOnly).get(name);
if (declaration) {
return this.follow(declaration, source, new Set(), typeOnly);
}
}
const declaration = this.declarations(source, typeOnly).get(name);
return declaration ? this.follow(declaration, source, new Set(), typeOnly) : undefined;
}
typeName(name: ts.EntityName, source: ts.SourceFile): DeclarationRef | undefined {
return this.reference(name, source, true);
}
reference(
name: ts.EntityName | ts.Expression,
source: ts.SourceFile,
typeOnly = false,
): DeclarationRef | undefined {
if (ts.isIdentifier(name)) {
return this.lookup(name.text, name, source, typeOnly);
}
if (!ts.isQualifiedName(name) && !ts.isPropertyAccessExpression(name)) {
return;
}
const left = this.reference(
ts.isQualifiedName(name) ? name.left : name.expression,
source,
typeOnly,
);
const right = ts.isQualifiedName(name) ? name.right : name.name;
if (!left) {
return;
}
if (ts.isNamespaceImport(left.node)) {
const declaration = left.node.parent.parent;
const imported = this.module(
(declaration.moduleSpecifier as ts.StringLiteral).text,
left.source,
);
return imported ? this.exported(right.text, imported, new Set(), typeOnly) : undefined;
}
if (
ts.isModuleDeclaration(left.node) &&
left.node.body &&
ts.isModuleBlock(left.node.body)
) {
const declaration = this.declarations(left.node.body, typeOnly).get(right.text);
return declaration ? { node: declaration, source: left.source } : undefined;
}
return;
}
private declarations(
scope: ts.SourceFile | ts.Block | ts.ModuleBlock,
typeOnly = false,
): Map<string, ts.Declaration> {
const cache = typeOnly ? this.typeNames : this.names;
let names = cache.get(scope);
if (names) {
return names;
}
names = new Map();
for (const statement of scope.statements) {
if (ts.isVariableStatement(statement)) {
if (typeOnly) {
continue;
}
for (const declaration of statement.declarationList.declarations) {
if (ts.isIdentifier(declaration.name)) {
names.set(declaration.name.text, declaration);
} else if (ts.isObjectBindingPattern(declaration.name)) {
for (const element of declaration.name.elements) {
if (ts.isIdentifier(element.name)) {
names.set(element.name.text, element);
}
}
}
}
} else if (ts.isImportDeclaration(statement)) {
const clause = statement.importClause;
if (clause?.name) {
names.set(clause.name.text, clause);
}
const bindings = clause?.namedBindings;
if (bindings && ts.isNamedImports(bindings)) {
for (const binding of bindings.elements) {
names.set(binding.name.text, binding);
}
} else if (bindings) {
names.set(bindings.name.text, bindings);
}
} else if (
'name' in statement &&
statement.name &&
ts.isIdentifier(statement.name as ts.Node)
) {
if (typeOnly && ts.isFunctionDeclaration(statement)) {
continue;
}
if (
!typeOnly &&
(ts.isTypeAliasDeclaration(statement) || ts.isInterfaceDeclaration(statement))
) {
continue;
}
names.set(
(statement.name as ts.Identifier).text,
statement as unknown as ts.Declaration,
);
}
}
cache.set(scope, names);
return names;
}
private follow(
node: ts.Declaration,
source: ts.SourceFile,
seen: Set<ts.Node>,
typeOnly = false,
): DeclarationRef | undefined {
if (seen.has(node)) {
return;
}
seen.add(node);
if (ts.isImportSpecifier(node)) {
const declaration = node.parent.parent.parent;
const imported = this.module(
(declaration.moduleSpecifier as ts.StringLiteral).text,
source,
);
if (imported) {
return this.exported(
(node.propertyName ?? node.name).text,
imported,
seen,
typeOnly,
);
}
} else if (ts.isImportClause(node)) {
const imported = this.module(
(node.parent.moduleSpecifier as ts.StringLiteral).text,
source,
);
if (imported) {
return this.exported('default', imported, seen, typeOnly);
}
}
return { node, source };
}
exported(
name: string,
source: ts.SourceFile,
seen = new Set<ts.Node>(),
typeOnly = false,
): DeclarationRef | undefined {
const local = this.declarations(source, typeOnly).get(name);
if (local) {
return this.follow(local, source, seen, typeOnly);
}
for (const statement of source.statements) {
if (name === 'default') {
if (
(ts.isClassDeclaration(statement) ||
ts.isFunctionDeclaration(statement) ||
ts.isInterfaceDeclaration(statement)) &&
statement.modifiers?.some(
(modifier) => modifier.kind === ts.SyntaxKind.DefaultKeyword,
)
) {
return { node: statement, source };
}
if (ts.isExportAssignment(statement) && ts.isIdentifier(statement.expression)) {
return this.lookup(
statement.expression.text,
statement.expression,
source,
typeOnly,
);
}
}
if (!ts.isExportDeclaration(statement) || seen.has(statement)) {
continue;
}
seen.add(statement);
const imported =
statement.moduleSpecifier && ts.isStringLiteral(statement.moduleSpecifier)
? this.module(statement.moduleSpecifier.text, source)
: source;
if (!imported) {
if (statement.exportClause && ts.isNamedExports(statement.exportClause)) {
const element = statement.exportClause.elements.find(
(element) => element.name.text === name,
);
if (element && runtimeImportName(element)) {
return { node: element, source };
}
}
continue;
}
if (!statement.exportClause) {
const ref = this.exported(name, imported, seen, typeOnly);
if (ref) {
return ref;
}
} else if (ts.isNamedExports(statement.exportClause)) {
const element = statement.exportClause.elements.find((el) => el.name.text === name);
if (element) {
return this.exported(
(element.propertyName ?? element.name).text,
imported,
seen,
typeOnly,
);
}
}
}
return;
}
}
export function fail(node: ts.Node, message: string): never {
const source = node.getSourceFile();
const position = source.getLineAndCharacterOfPosition(node.getStart(source));
throw new Error(
`[${packageName}] ${source.fileName}:${position.line + 1}:${position.character + 1}: ${message}`,
);
}
+43
View File
@@ -0,0 +1,43 @@
import { existsSync, readFileSync } from 'node:fs';
import { resolve } from 'node:path';
import type { ResolvedConfig } from 'vite';
export function buildEntries(config: ResolvedConfig): string[] {
const input =
config.build.rolldownOptions.input ??
(config.build.lib && config.build.lib.entry) ??
'index.html';
const files =
typeof input === 'string' ? [input] : Array.isArray(input) ? input : Object.values(input);
return files.flatMap((file) => {
const path = resolve(config.root, file);
if (!existsSync(path)) {
return [];
}
if (/\.[cm]?[jt]sx?$/.test(path)) {
return [path];
}
if (!path.endsWith('.html')) {
return [];
}
return [
...readFileSync(path, 'utf8').matchAll(
/<script\b[^>]*\bsrc\s*=\s*["']([^"']+)["'][^>]*>/gi,
),
].flatMap(([, src]) => {
if (/^(?:[a-z]+:)?\/\//i.test(src)) {
return [];
}
const entry = resolve(
src.startsWith('/') ? config.root : resolve(path, '..'),
src.replace(/^\//, '').split('?')[0],
);
return /\.[cm]?[jt]sx?$/.test(entry) && existsSync(entry) ? [entry] : [];
});
});
}
@@ -0,0 +1,242 @@
import { describe, expect, it } from 'vitest';
import { run, imports } from '../upstream/compile';
import { bundle } from '../upstream/project';
import { schemaCodecs, schemaRegistry } from '../upstream/schema';
import { generateCode } from '../../src/codegen/generator';
import { protobuf_getUnknownFields } from '../../src/runtime';
import { ProtobufCompiler } from '../../src/compiler/compiler';
import { withProject } from '../upstream/project';
import { join } from 'node:path';
describe('SnowLuma compatibility', () => {
it('leaves generic data unions and nested business types outside codec discovery', () => {
expect(
run(`
type Json = string | Json[];
interface Field<T> {value:T; metadata:Json;}
type Wrapper<T> = Field<T> & { extra?:Json };
function identity<T>(value:Field<T>) {return value;}
function generic<T>(field:Field<Wrapper<T>>) {return identity(field);}
function result<T>(ok:boolean, value:T) {if(ok) return {ok:true, value}; return {ok:false, value};}
const field:Field<Wrapper<string>> = {value:{value:'ok', metadata:[]}, metadata:[]};
export const output = result(true, generic(field)).value.value.value;
`).output,
).toBe('ok');
});
it('keeps unrelated callback aliases and inferred containers lazy', () => {
expect(
run(`
type Json = string | Json[];
type Handler<T> = (value:T) => void;
interface Field<T> {value:T; metadata:Json;}
const field:Field<Json> = {value:'ok', metadata:[]};
function wrap<T>(value:Field<T>) {return {get(){return value.value;}};}
const handler:Handler<Json> = (value) => {};
handler('x');
export const result = wrap(field).get();
`).result,
).toBe('ok');
});
it('propagates composite class bindings through generic factories', () => {
expect(
run(`
interface Box<T> { value:pb<1,T>; }
class Codec<T> { encode(value:T) { return protobuf_encode<T>(value); } }
function create<T>() { return new Codec<Box<T>>(); }
export const result = [...create<uint_32>().encode({value:7})];
`).result,
).toEqual([8, 7]);
});
it('defers unrelated class type arguments but validates codec class schemas', () => {
expect(
run(`
type Json = string | Json[];
class Field<T> { value(): null { return null; } }
function field() { return new Field<Json>(); }
export const result = field().value();
`).result,
).toBeNull();
expect(() =>
run(`
class Codec<T> { encode(value:T) {return protobuf_encode<T>(value);} }
new Codec<Missing>().encode({});
`),
).toThrow(/Unresolved type Missing/);
});
it('infers matching generic containers without expanding unrelated recursive data', () => {
expect(
run(`
type Json = string | Json[];
interface Field<T> { value: T; metadata: Json; }
function arrayField<E>(field: Field<E>) { return Object.assign({}, { get: () => field.value }); }
function wrap<E>(field: Field<E>) { return arrayField(field); }
const field: Field<string> = { value:'ok', metadata:[] };
export const result = wrap(field).get();
`).result,
).toBe('ok');
});
it('does not resolve recursive data aliases while looking for class receivers', () => {
expect(
run(`
type Data<T> = { children: T[]; name: string };
type Element = Data<Element>;
type Wrapped<T> = T;
function data(): Wrapped<Element> { return { children: [], name: 'ok' }; }
export const result = data().name.toUpperCase();
`).result,
).toBe('OK');
});
it('does not resolve ordinary generic return values or data alias members as callbacks', () => {
expect(
run(`
type Id<T> = T;
interface Data { value: string; children: Id<Data>[]; }
const data: Id<Data> = {value:'ok', children:[]};
function decode<T>(raw: Uint8Array, callback: () => T): T | null { return callback(); }
const raw: Uint8Array = new Uint8Array();
const decoded = decode(raw, () => data);
export const result = [decoded.value, data.value];
`).result,
).toEqual(['ok', 'ok']);
});
it('does not specialize ambient library functions', async () => {
const source = `import { expect } from './external';
const data: Record<string, unknown> = {};
expect(data).check();`;
await withProject(
{
'external.d.ts': 'export declare function expect<T>(value: T): { check(): void };',
'entry.ts': source,
},
(directory) => {
expect(
new ProtobufCompiler().compile(source, join(directory, 'entry.ts')),
).toBeNull();
},
);
});
it('leaves ordinary calls and data receivers outside schema resolution', () => {
expect(
run(`
interface Options { meta?: Record<string, unknown>; }
function options(value: Options): Options { return value; }
const input: Options = { meta: { source: 'test' } };
const copy = options(input);
function data(flag: boolean) { if (flag) return { value: 7 }; return { value: 8 }; }
function bytes(value: Uint8Array): Uint8Array { return value; }
const raw: Uint8Array = new Uint8Array([1]);
export const result = [copy.meta.source, data(true).value, bytes(raw).length];
`).result,
).toEqual(['test', 7, 1]);
});
it('keeps TypeScript optionality separate from explicit wire presence', () => {
const result = run(`
interface Msg {
plain?: pb<1, uint_32>;
flag?: pb<2, bool>;
text?: pb<3, string>;
data?: pb<4, bytes>;
big?: pb<5, uint_64>;
forced?: pb_optional<6, uint_32>;
}
export const wire = [...protobuf_encode<Msg>({
plain: 0, flag: false, text: '', data: new Uint8Array(), big: 0n, forced: 0,
})];
export const decoded = protobuf_decode<Msg>(new Uint8Array(wire));
`);
expect(result.wire).toEqual([48, 0]);
expect(result.decoded).toEqual({
plain: null,
flag: null,
text: null,
data: null,
big: null,
forced: 0,
});
});
it('retains aliased markers and presence through exported generic wrappers', async () => {
const result = await bundle({
'schema.ts': `import type { pb as P, pb_optional as O, pb_repeated as R, uint_32 } from '@snowluma/proton';
export interface Box<T> { value?: P<1, T>; }
export interface Msg { plain?: P<1, uint_32>; forced?: O<2, uint_32>; items?: R<3, uint_32>; }
export interface Outer<U> { inner?: P<1, Box<U>>; }`,
'barrel.ts': `export * from './schema';`,
'wrapper.ts': `${imports} import type {Outer} from './barrel';
export function encode<T>(value:Outer<T>) { return protobuf_encode<Outer<T>>(value); }`,
'forward.ts': `import {encode} from './wrapper'; import type {Outer} from './barrel';
export function forward<T>(value:Outer<T>) { return encode<T>(value); }`,
'entry.ts': `${imports} import type {Msg} from './barrel'; import {forward} from './forward';
export const wire = [...forward<Msg>({inner:{value:{plain:0,forced:0,items:[0,7]}}})];`,
});
expect(result.wire).toEqual([10, 8, 10, 6, 16, 0, 24, 0, 24, 7]);
});
it.each([true, false])('resolves package exports with tsconfig=%s', async (withConfig) => {
const result = await bundle({
...(withConfig
? {
'tsconfig.json': JSON.stringify({
compilerOptions: { moduleResolution: 'bundler', module: 'esnext' },
}),
}
: {}),
'node_modules/@fake/schemas/package.json': JSON.stringify({
name: '@fake/schemas',
type: 'module',
exports: { '.': { types: './index.ts', default: './index.ts' } },
}),
'node_modules/@fake/schemas/index.ts': `export * from './schema';`,
'node_modules/@fake/schemas/schema.ts': `export interface Msg { value:pb<1,uint_32>; }`,
'entry.ts': `${imports} import type {Msg} from '@fake/schemas';
export const result = protobuf_decode<Msg>(protobuf_encode<Msg>({value:42}));`,
});
expect(result).toEqual({ result: { value: 42 } });
});
const codec = schemaCodecs<{ value: number }>('interface Msg { value:pb<1,uint_32>; }');
it.each([
['unterminated group', [0x13, 8, 7]],
['mismatched group', [0x13, 0x1c]],
['unexpected group end', [0x14]],
['invalid group tag', [0x13, 0]],
['group payload exceeds frame', [0x13, 0x1a, 4, 0x14]],
['group recursion limit', [...Array(66).fill(0x13), ...Array(66).fill(0x14)]],
] as const)('rejects %s', (_name, bytes) => {
expect(() => codec.decode(Uint8Array.from(bytes))).toThrow(/protobuf/);
});
it('skips nested groups and retains non-enumerable metadata', () => {
const decoded = codec.decode(Uint8Array.from([0x13, 0x1b, 8, 99, 0x1c, 0x14, 8, 7]));
expect(decoded).toEqual({ value: 7 });
expect(protobuf_getUnknownFields(decoded)).toEqual([
{ fieldNumber: 2, wireType: 3, count: 1, totalByteLength: 5 },
]);
expect(JSON.stringify(decoded)).toBe('{"value":7}');
expect([...codec.encode(decoded)]).toEqual([8, 7]);
});
it.each([
[NaN, 0],
[0, NaN],
[0.5, 1],
[0, 0.5],
[-1, 1],
[1, 0],
[0, 3],
[0, Infinity],
])('rejects invalid decoder bounds %s..%s', (offset, end) => {
const generated = generateCode(schemaRegistry('interface Msg { value:pb<1,uint_32>; }'));
const decode = new Function(generated + '\nreturn protobuf_decode_Msg;')();
expect(() => decode(new Uint8Array([8, 1]), offset, end)).toThrow(/bounds/);
});
});
@@ -0,0 +1,342 @@
import { describe, it, expect } from 'vitest';
import { schemaRegistry as analyzeSource } from '../upstream/schema';
import { generateCode } from '../../src/codegen/generator';
import { protobuf_getUnknownFieldMetadata } from '../../src/runtime';
import { loadFixture } from '../helpers';
/** Analyze fixture → generate → eval → return encode/decode pair */
function makeRoundTrip(fixtureName: string, msgName: string) {
return makeRoundTripFromSchema(loadFixture(fixtureName), msgName);
}
function makeRoundTripFromSchema(schema: string, msgName: string) {
const gen = generateCode(analyzeSource(schema, `${msgName}.ts`));
return new Function(
gen + `\nreturn { enc: protobuf_encode_${msgName}, dec: protobuf_decode_${msgName} };`,
)() as {
enc: (obj: unknown) => Uint8Array;
dec: (data: Uint8Array) => any;
};
}
function encodeVarint32(value: number): number[] {
const bytes: number[] = [];
let remaining = value >>> 0;
do {
let byte = remaining & 0x7f;
remaining >>>= 7;
if (remaining !== 0) {
byte |= 0x80;
}
bytes.push(byte);
} while (remaining !== 0);
return bytes;
}
describe('code generator', () => {
it.each([
[0, []],
[1, [0x08, 0x01]],
[127, [0x08, 0x7f]],
[128, [0x08, 0x80, 0x01]],
[300, [0x08, 0xac, 0x02]],
[16384, [0x08, 0x80, 0x80, 0x01]],
[0xffffffff, [0x08, 0xff, 0xff, 0xff, 0xff, 0x0f]],
] as const)('encodes and decodes uint32 wire fixture %i', (value, bytes) => {
const { enc, dec } = makeRoundTrip('simple.ts', 'SimpleMsg');
expect([...enc({ value })]).toEqual(bytes);
expect(dec(Uint8Array.from(bytes)).value).toBe(value);
});
it.each([
[0x10, 0xac, 0x02],
[0x12, 0x03, 0x41, 0x42, 0x43],
[0x15, 0, 0, 0, 0],
[0x11, 0, 0, 0, 0, 0, 0, 0, 0],
])('skips an unknown wire field before a known field (%j)', (...bytes) => {
const { dec } = makeRoundTrip('simple.ts', 'SimpleMsg');
expect(dec(Uint8Array.from([...bytes, 0x08, 42])).value).toBe(42);
});
it('generates encode/decode for simple uint_32', () => {
const gen = generateCode(analyzeSource(loadFixture('simple.ts'), 't.ts'));
expect(gen).toContain('function protobuf_encode_SimpleMsg(obj)');
expect(gen).toContain('function protobuf_decode_SimpleMsg(data');
// Tag for field 1, varint should be pre-computed as 0x08
expect(gen).toContain('buf[offset++] = 8;');
});
it('generates nested message code', () => {
const gen = generateCode(analyzeSource(loadFixture('nested.ts'), 't.ts'));
expect(gen).toContain('__write_Inner(');
expect(gen).toContain('protobuf_decode_Inner(data, offset, offset + _len)');
});
it('round-trip uint_32', () => {
const { enc, dec } = makeRoundTrip('simple.ts', 'SimpleMsg');
expect(dec(enc({ value: 42 })).value).toBe(42);
});
it('round-trip nested message', () => {
const { enc, dec } = makeRoundTrip('nested.ts', 'Outer');
expect(dec(enc({ inner: { value: 999 } })).inner.value).toBe(999);
});
it('round-trip string', () => {
const { enc, dec } = makeRoundTrip('string-msg.ts', 'StringMsg');
expect(dec(enc({ text: 'hello world' })).text).toBe('hello world');
});
it('round-trip unicode string', () => {
const { enc, dec } = makeRoundTrip('string-msg.ts', 'StringMsg');
expect(dec(enc({ text: '你好,protobuf🚀' })).text).toBe('你好,protobuf🚀');
});
it('round-trip bool', () => {
const { enc, dec } = makeRoundTrip('bool-msg.ts', 'BoolMsg');
expect(dec(enc({ active: true })).active).toBe(true);
const empty = enc({ active: false });
expect(empty.length).toBe(0);
expect(dec(empty).active).toBe(false);
});
it('proto3 default encoding (0 → empty)', () => {
const { enc } = makeRoundTrip('simple.ts', 'SimpleMsg');
expect(enc({ value: 0 }).length).toBe(0);
});
it('round-trip repeated uint_32', () => {
const { enc, dec } = makeRoundTrip('repeated.ts', 'RepeatedMsg');
const result = dec(enc({ ids: [10, 20, 30], names: [] }));
expect(result.ids).toEqual([10, 20, 30]);
expect(result.names).toEqual([]);
});
it('round-trip repeated string', () => {
const { enc, dec } = makeRoundTrip('repeated.ts', 'RepeatedMsg');
const result = dec(enc({ ids: [], names: ['hello', 'world'] }));
expect(result.ids).toEqual([]);
expect(result.names).toEqual(['hello', 'world']);
});
it('repeated empty arrays round-trip', () => {
const { enc, dec } = makeRoundTrip('repeated.ts', 'RepeatedMsg');
const result = dec(enc({ ids: [], names: [] }));
expect(result.ids).toEqual([]);
expect(result.names).toEqual([]);
});
it('round-trip bigint-backed 64-bit integers', () => {
const schema = `
interface BigIntMsg {
unsigned: pb<1, uint_64>;
signed: pb<2, int_64>;
zigzag: pb<3, sint_64>;
fixed: pb<4, fixed_64>;
sfixed: pb<5, sfixed_64>;
ids: pb_repeated<6, uint_64>;
}`;
const { enc, dec } = makeRoundTripFromSchema(schema, 'BigIntMsg');
const input = {
unsigned: 18446744073709551615n,
signed: -1n,
zigzag: -1234567890123456789n,
fixed: 0x0102030405060708n,
sfixed: -0x0102030405060708n,
ids: [1n, 2n, 9007199254740993n],
};
expect(dec(enc(input))).toEqual(input);
});
it('round-trip sint_32 with negative values', () => {
const schema = `
interface Sint32Msg {
value: pb<1, sint_32>;
values: pb_repeated<2, sint_32>;
}`;
const { enc, dec } = makeRoundTripFromSchema(schema, 'Sint32Msg');
const input = { value: -789, values: [-1, 0, 1, -789, 2147483647, -2147483648] };
expect(dec(enc(input))).toEqual(input);
});
it('preserves legacy outbound int_32 bytes while accepting canonical negative varints', () => {
const schema = `
interface Int32Msg {
value: pb<1, int_32>;
values: pb_repeated<2, int_32>;
forced: pb_optional<3, int_32>;
}`;
const { enc, dec } = makeRoundTripFromSchema(schema, 'Int32Msg');
const input = {
value: -1,
values: [-2147483648, -1, 0, 1, 2147483647],
forced: -2147483648,
};
const encoded = enc(input);
// Keep SnowLuma's established five-byte negative int32 encoding on the
// outbound path until QQ interoperability is verified on a live account.
// The decoder remains liberal and accepts protobuf's canonical ten-byte
// negative form below.
expect(Array.from(encoded.slice(0, 6))).toEqual([0x08, 0xff, 0xff, 0xff, 0xff, 0x0f]);
expect(dec(encoded)).toEqual(input);
expect(
dec(Uint8Array.from([0x08, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x01]))
.value,
).toBe(-1);
});
it('retains the low 32 bits of extended uint_32 field varints', () => {
const { dec } = makeRoundTripFromSchema(
`interface Uint32Msg { value: pb<1, uint_32>; }`,
'Uint32Msg',
);
// QQ sometimes serializes a signed 32-bit source into a schema slot that
// clients expose as uint32. Protobuf readers consume the full ten-byte
// varint and retain its low 32 bits for wire compatibility.
expect(
dec(Uint8Array.from([0x08, 0xfe, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x01]))
.value,
).toBe(0xfffffffe);
// QQ-compatible uint32 readers consume up to a full 64-bit varint and
// retain its low 32 bits. Some push-envelope sequences use that wire
// representation, so values with non-zero high bits must remain decodable.
expect(dec(Uint8Array.from([0x08, 0x80, 0x80, 0x80, 0x80, 0x10])).value).toBe(0);
expect(dec(Uint8Array.from([0x08, 0x87, 0x80, 0x80, 0x80, 0x10])).value).toBe(7);
// Compatibility applies only to complete uint64 varints. Truncated input
// and values wider than 64 bits must still fail loudly.
expect(() => dec(Uint8Array.from([0x08, 0x80]))).toThrow(/[Tt]runcated.*varint/);
expect(() =>
dec(
Uint8Array.from([0x08, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x02]),
),
).toThrow(/Invalid protobuf varint/);
});
it('rejects malformed length-delimited fields without leaving parent bounds', () => {
const schema = `interface BytesMsg { data: pb<1, bytes>; }`;
const { dec } = makeRoundTripFromSchema(schema, 'BytesMsg');
expect(() => dec(Uint8Array.from([0x0a, 0x80]))).toThrow(/[Tt]runcated.*varint/);
expect(() => dec(Uint8Array.from([0x0a, 0x80, 0x80, 0x80, 0x80, 0x10]))).toThrow(
/Invalid protobuf varint/,
);
});
it('skips unknown groups without decoding their fields into the parent', () => {
const schema = `
interface Text { str: pb<1, string>; }
interface Elem { text: pb<1, Text>; }
`;
const { dec } = makeRoundTripFromSchema(schema, 'Elem');
const decoded = dec(
Uint8Array.from([
0x7b, 0x0a, 0x08, 0x0a, 0x06, 0x66, 0x6f, 0x72, 0x67, 0x65, 0x64, 0x7c,
]),
);
expect(decoded.text).toBeNull();
expect(protobuf_getUnknownFieldMetadata(decoded).fields).toMatchObject([
{ fieldNumber: 15, wireType: 3 },
]);
});
it('aggregates repeated unknown fields instead of retaining every occurrence', () => {
const { dec } = makeRoundTripFromSchema(
`interface UnknownMsg { known: pb<1, uint_32>; }`,
'UnknownMsg',
);
const wire: number[] = [];
const tag = encodeVarint32((60 << 3) | 0);
for (let index = 0; index < 1_000; index++) {
wire.push(...tag, 0);
}
const metadata = protobuf_getUnknownFieldMetadata(dec(Uint8Array.from(wire)));
expect(metadata).toEqual({
fields: [{ fieldNumber: 60, wireType: 0, count: 1_000, totalByteLength: 1_000 }],
totalOccurrences: 1_000,
omittedOccurrences: 0,
omittedByteLength: 0,
});
});
it('caps retained unknown field kinds and summarizes overflow', () => {
const { dec } = makeRoundTripFromSchema(
`interface UnknownKindsMsg { known: pb<1, uint_32>; }`,
'UnknownKindsMsg',
);
const wire: number[] = [];
for (let fieldNumber = 2; fieldNumber <= 66; fieldNumber++) {
wire.push(...encodeVarint32((fieldNumber << 3) | 0), 0);
}
const metadata = protobuf_getUnknownFieldMetadata(dec(Uint8Array.from(wire)));
expect(metadata.fields).toHaveLength(64);
expect(metadata.totalOccurrences).toBe(65);
expect(metadata.omittedOccurrences).toBe(1);
expect(metadata.omittedByteLength).toBe(1);
});
it('round-trip double as ieee754 number', () => {
const schema = `
interface DoubleMsg {
value: pb<1, double>;
values: pb_repeated<2, double>;
}`;
const { enc, dec } = makeRoundTripFromSchema(schema, 'DoubleMsg');
const result = dec(enc({ value: Math.PI, values: [Math.E, -0.5] }));
expect(result.value).toBeCloseTo(Math.PI);
expect(result.values[0]).toBeCloseTo(Math.E);
expect(result.values[1]).toBeCloseTo(-0.5);
});
});
describe('pb_optional explicit presence', () => {
it('serialises a zero scalar (where pb<> would omit it)', () => {
const { enc, dec } = makeRoundTripFromSchema(
`interface OptMsg { forced: pb_optional<1, uint_32>; }`,
'OptMsg',
);
// field 1, varint, value 0 → tag 0x08 then payload 0x00 (NOT empty).
expect(Array.from(enc({ forced: 0 }))).toEqual([0x08, 0x00]);
expect(dec(enc({ forced: 0 })).forced).toBe(0);
});
it('still omits the field when the value is absent (undefined)', () => {
const { enc } = makeRoundTripFromSchema(
`interface OptMsg2 { forced: pb_optional<1, uint_32>; }`,
'OptMsg2',
);
expect(enc({}).length).toBe(0);
});
it('does not change a sibling plain pb<> field (it still omits its zero)', () => {
const { enc, dec } = makeRoundTripFromSchema(
`interface MixMsg { plain: pb<1, uint_32>; forced: pb_optional<2, uint_32>; }`,
'MixMsg',
);
// plain=0 omitted (unchanged); only forced=0 is emitted (field 2 → tag 0x10).
expect(Array.from(enc({ plain: 0, forced: 0 }))).toEqual([0x10, 0x00]);
expect(dec(enc({ plain: 7, forced: 9 }))).toEqual({ plain: 7, forced: 9 });
});
it('forces a false bool onto the wire', () => {
const { enc, dec } = makeRoundTripFromSchema(
`interface BoolOpt { flag: pb_optional<1, bool>; }`,
'BoolOpt',
);
expect(Array.from(enc({ flag: false }))).toEqual([0x08, 0x00]);
expect(dec(enc({ flag: false })).flag).toBe(false);
expect(Array.from(enc({ flag: true }))).toEqual([0x08, 0x01]);
});
});
+3
View File
@@ -0,0 +1,3 @@
interface BoolMsg {
active: pb<1, bool>;
}
+7
View File
@@ -0,0 +1,7 @@
interface Inner {
value: pb<1, uint_32>;
}
interface Outer {
inner: pb<1, Inner>;
}
+9
View File
@@ -0,0 +1,9 @@
interface RepeatedMsg {
ids: pb_repeated<1, uint_32>;
names: pb_repeated<2, string>;
}
const data = protobuf_encode<RepeatedMsg>({ ids: [1, 2, 3], names: ["a", "b"] });
const decoded = protobuf_decode<RepeatedMsg>(data);
export { };
+3
View File
@@ -0,0 +1,3 @@
interface SimpleMsg {
value: pb<1, uint_32>;
}
+3
View File
@@ -0,0 +1,3 @@
interface StringMsg {
text: pb<1, string>;
}
+10
View File
@@ -0,0 +1,10 @@
import { readFileSync } from 'fs';
import { dirname, resolve } from 'path';
import { fileURLToPath } from 'url';
const fixturesDir = resolve(dirname(fileURLToPath(import.meta.url)), '..', 'fixtures');
/** Load a fixture file as a string. */
export function loadFixture(name: string): string {
return readFileSync(resolve(fixturesDir, name), 'utf-8');
}
@@ -0,0 +1,94 @@
import { describe, expect, it } from 'vitest';
import { schemaCodecs } from '../schema';
const bytesSchema = 'interface Msg { data: pb<1,bytes>; }';
describe('encoded buffers', () => {
it('round-trips messages around allocation and slab boundaries', () => {
const codec = schemaCodecs<{ data: Uint8Array }>(bytesSchema);
for (const length of [
0, 1, 61, 62, 63, 64, 65, 125, 127, 128, 4092, 4093, 4094, 4095, 4096, 8192,
]) {
const data = Uint8Array.from({ length }, (_, i) => (i * 13) & 255);
const encoded = codec.encode({ data });
expect(codec.decode(encoded)).toEqual({ data });
if (encoded.length <= 64 || encoded.length > 4096) {
expect(encoded.byteOffset).toBe(0);
expect(encoded.buffer.byteLength).toBe(encoded.length);
}
}
});
it('keeps retained results intact across slab rotation and subsequent mutations', () => {
const codec = schemaCodecs<{ data: Uint8Array }>(bytesSchema);
const retained = Array.from({ length: 200 }, (_, index) => {
const data = new Uint8Array([63, 128, 800, 1300][index % 4]).fill(index);
const encoded = codec.encode({ data });
return { data, encoded, snapshot: encoded.slice() };
});
expect(retained.some(({ encoded }) => encoded.byteOffset > 0)).toBe(true);
for (const { data, encoded, snapshot } of retained) {
expect(encoded).toEqual(snapshot);
expect(codec.decode(encoded)).toEqual({ data });
}
retained[0].encoded.fill(0);
for (const { encoded, snapshot } of retained.slice(1)) {
expect(encoded).toEqual(snapshot);
}
});
it('supports encoding from a getter without reusing the inner result', () => {
const codec = schemaCodecs<{ data: Uint8Array }>(bytesSchema);
const inner = new Uint8Array(128).fill(7);
const outer = new Uint8Array(256).fill(9);
let nested: Uint8Array | undefined;
const encoded = codec.encode({
get data() {
nested = codec.encode({ data: inner });
return outer;
},
});
expect(codec.decode(encoded)).toEqual({ data: outer });
expect(codec.decode(nested!)).toEqual({ data: inner });
});
it('replaces a slab whose backing buffer was transferred', () => {
const codec = schemaCodecs<{ data: Uint8Array }>(bytesSchema);
const data = new Uint8Array(128).fill(3);
const first = codec.encode({ data });
const transferred = structuredClone(first, { transfer: [first.buffer as ArrayBuffer] });
expect(first.byteLength).toBe(0);
expect(codec.decode(transferred)).toEqual({ data });
expect(codec.decode(codec.encode({ data }))).toEqual({ data });
});
it('decodes unaligned pooled messages containing strings, fixed values and nested data', () => {
const codec = schemaCodecs<{
text: string;
ratio: number;
inner: { value: bigint; data: Uint8Array };
}>(`
interface Inner {value:pb<1,fixed_64>;data:pb<2,bytes>;}
interface Msg {text:pb<1,string>;ratio:pb<2,double>;inner:pb<3,Inner>;}
`);
let unaligned = false;
for (let i = 0; i < 100; i++) {
const value = {
text: '中文🚀'.repeat(8) + i,
ratio: Math.PI + i,
inner: { value: (1n << 63n) + BigInt(i), data: new Uint8Array(17 + i).fill(i) },
};
const encoded = codec.encode(value);
unaligned ||= encoded.byteOffset % 8 !== 0;
expect(codec.decode(encoded)).toEqual(value);
}
expect(unaligned).toBe(true);
});
});
@@ -0,0 +1,245 @@
import { describe, it, expect } from 'vitest';
import { schemaRegistry } from '../schema';
import { generateCode } from '../../../src/codegen/generator';
const simple = 'interface SimpleMsg { value: pb<1, uint_32>; }';
const nested =
'interface Inner { value: pb<1, uint_32>; } interface Outer { inner: pb<1, Inner>; }';
const string = 'interface StringMsg { text: pb<1, string>; }';
const bool = 'interface BoolMsg { active: pb<1, bool>; }';
const repeated =
'interface RepeatedMsg { ids: pb_repeated<1, uint_32>; names: pb_repeated<2, string>; }';
function makeRoundTrip(schema: string, msgName: string) {
const gen = generateCode(schemaRegistry(schema, `${msgName}.ts`));
return new Function(
gen + `\nreturn { enc: protobuf_encode_${msgName}, dec: protobuf_decode_${msgName} };`,
)() as {
enc: (obj: any) => Uint8Array;
dec: (data: Uint8Array) => any;
};
}
describe('code generator', () => {
it('allocates one output buffer for nested repeated messages', () => {
const schema = `interface Leaf { text: pb<1,string>; } interface Msg { children: pb_repeated<1,Leaf>; }`;
const registry = schemaRegistry(schema, 'allocation.ts');
const code = generateCode(registry, { encodeRoots: ['Msg'], decodeRoots: [] });
let allocations = 0;
class CountingBytes extends Uint8Array {
constructor(size: number) {
super(size);
allocations++;
}
static get [Symbol.species]() {
return Uint8Array;
}
}
const encode = new Function('Uint8Array', code + '\nreturn protobuf_encode_Msg;')(
CountingBytes,
);
const input = { children: Array.from({ length: 32 }, (_, i) => ({ text: '你好' + i })) };
const encoded = encode(input);
expect(allocations).toBe(1);
expect(makeRoundTrip(schema, 'Msg').dec(encoded)).toEqual(input);
});
it('omits unused helpers and decoder functions from encode-only output', () => {
const code = generateCode(
schemaRegistry('interface Msg { value: pb<1,uint_32>; }', 'minimal.ts'),
{
encodeRoots: ['Msg'],
decodeRoots: [],
},
);
expect(code).not.toContain('TextDecoder');
expect(code).not.toContain('TextEncoder');
expect(code).not.toContain('DataView');
expect(code).not.toContain('BigInt');
expect(code).not.toContain('protobuf_decode_');
});
it('round-trips a deep mixture of strings and nested messages', () => {
const schema = Array.from(
{ length: 24 },
(_, i) =>
`interface M${i} { text: pb<1,string>; ${i < 23 ? `next: pb<2,M${i + 1}>;` : ''} }`,
).join('\n');
let input: any = { text: 'end' };
for (let i = 22; i >= 0; i--) {
input = { text: 'level' + i, next: input };
}
const { enc, dec } = makeRoundTrip(schema, 'M0');
expect(dec(enc(input))).toEqual(input);
});
it('round-trips int_32 and preserves five-byte negative varints', () => {
const { enc, dec } = makeRoundTrip(
`interface IntMsg {
value: pb<1, int_32>;
values: pb_repeated<2, int_32>;
}`,
'IntMsg',
);
expect([...enc({ value: -1 })]).toEqual([0x08, 0xff, 0xff, 0xff, 0xff, 0x0f]);
const input = { value: -2147483648, values: [-1, -2147483648, 0, 2147483647] };
expect(dec(enc(input))).toEqual(input);
});
it('preserves fixed_32 unsigned values and sfixed_32 signed values', () => {
const { enc, dec } = makeRoundTrip(
`interface FixedMsg {
unsigned: pb<1, fixed_32>;
signed: pb<2, sfixed_32>;
values: pb_repeated<3, fixed_32>;
}`,
'FixedMsg',
);
const input = {
unsigned: 0xffffffff,
signed: -2147483648,
values: [0x80000000, 0xffffffff],
};
expect(dec(enc(input))).toEqual(input);
});
it('uses bigint defaults for all 64-bit integer fields', () => {
const { enc, dec } = makeRoundTrip(
`interface Defaults {
unsigned: pb<1, uint_64>;
signed: pb<2, int_64>;
zigzag: pb<3, sint_64>;
fixed: pb<4, fixed_64>;
sfixed: pb<5, sfixed_64>;
}`,
'Defaults',
);
const result = dec(new Uint8Array());
expect(result).toEqual({ unsigned: 0n, signed: 0n, zigzag: 0n, fixed: 0n, sfixed: 0n });
expect(enc(result)).toEqual(new Uint8Array());
});
it('decodes tags with the highest valid field number', () => {
const { enc, dec } = makeRoundTrip(
'interface HighTag { value: pb<536870911, uint_32>; }',
'HighTag',
);
const bytes = new Uint8Array([0xf8, 0xff, 0xff, 0xff, 0x0f, 7]);
expect(enc({ value: 7 })).toEqual(bytes);
expect(dec(bytes)).toEqual({ value: 7 });
});
it('generates encode/decode for simple uint_32', () => {
const gen = generateCode(schemaRegistry(simple, 't.ts'));
expect(gen).toContain('function protobuf_encode_SimpleMsg(obj)');
expect(gen).toContain('function protobuf_decode_SimpleMsg(data');
// Tag for field 1, varint should be pre-computed as 0x08
expect(gen).toContain('buf[offset++] = 8;');
});
it('generates nested message code', () => {
const gen = generateCode(schemaRegistry(nested, 't.ts'));
expect(gen).toContain('protobuf_encode_Inner(');
expect(gen).toContain('protobuf_decode_Inner(data, offset, offset + _len)');
});
it('round-trip uint_32', () => {
const { enc, dec } = makeRoundTrip(simple, 'SimpleMsg');
expect(dec(enc({ value: 42 })).value).toBe(42);
});
it('round-trip nested message', () => {
const { enc, dec } = makeRoundTrip(nested, 'Outer');
expect(dec(enc({ inner: { value: 999 } })).inner.value).toBe(999);
});
it('round-trip string', () => {
const { enc, dec } = makeRoundTrip(string, 'StringMsg');
expect(dec(enc({ text: 'hello world' })).text).toBe('hello world');
});
it('round-trip unicode string', () => {
const { enc, dec } = makeRoundTrip(string, 'StringMsg');
expect(dec(enc({ text: '你好,protobuf🚀' })).text).toBe('你好,protobuf🚀');
});
it('round-trip bool', () => {
const { enc, dec } = makeRoundTrip(bool, 'BoolMsg');
expect(dec(enc({ active: true })).active).toBe(true);
const empty = enc({ active: false });
expect(empty.length).toBe(0);
expect(dec(empty).active).toBe(false);
});
it('proto3 default encoding (0 → empty)', () => {
const { enc } = makeRoundTrip(simple, 'SimpleMsg');
expect(enc({ value: 0 }).length).toBe(0);
});
it('round-trip repeated uint_32', () => {
const { enc, dec } = makeRoundTrip(repeated, 'RepeatedMsg');
const result = dec(enc({ ids: [10, 20, 30], names: [] }));
expect(result.ids).toEqual([10, 20, 30]);
expect(result.names).toEqual([]);
});
it('round-trip repeated string', () => {
const { enc, dec } = makeRoundTrip(repeated, 'RepeatedMsg');
const result = dec(enc({ ids: [], names: ['hello', 'world'] }));
expect(result.ids).toEqual([]);
expect(result.names).toEqual(['hello', 'world']);
});
it('repeated empty arrays round-trip', () => {
const { enc, dec } = makeRoundTrip(repeated, 'RepeatedMsg');
const result = dec(enc({ ids: [], names: [] }));
expect(result.ids).toEqual([]);
expect(result.names).toEqual([]);
});
it('round-trip bigint-backed 64-bit integers', () => {
const schema = `
interface BigIntMsg {
unsigned: pb<1, uint_64>;
signed: pb<2, int_64>;
zigzag: pb<3, sint_64>;
fixed: pb<4, fixed_64>;
sfixed: pb<5, sfixed_64>;
ids: pb_repeated<6, uint_64>;
}`;
const { enc, dec } = makeRoundTrip(schema, 'BigIntMsg');
const input = {
unsigned: 18446744073709551615n,
signed: -1n,
zigzag: -1234567890123456789n,
fixed: 0x0102030405060708n,
sfixed: -0x0102030405060708n,
ids: [1n, 2n, 9007199254740993n],
};
expect(dec(enc(input))).toEqual(input);
});
it('round-trip sint_32 with negative values', () => {
const schema = `
interface Sint32Msg {
value: pb<1, sint_32>;
values: pb_repeated<2, sint_32>;
}`;
const { enc, dec } = makeRoundTrip(schema, 'Sint32Msg');
const input = { value: -789, values: [-1, 0, 1, -789, 2147483647, -2147483648] };
expect(dec(enc(input))).toEqual(input);
});
it('round-trip double as ieee754 number', () => {
const schema = `
interface DoubleMsg {
value: pb<1, double>;
values: pb_repeated<2, double>;
}`;
const { enc, dec } = makeRoundTrip(schema, 'DoubleMsg');
const result = dec(enc({ value: Math.PI, values: [Math.E, -0.5] }));
expect(result.value).toBeCloseTo(Math.PI);
expect(result.values[0]).toBeCloseTo(Math.E);
expect(result.values[1]).toBeCloseTo(-0.5);
});
});
@@ -0,0 +1,212 @@
import { deepStrictEqual } from 'node:assert';
import { describe, it, expect } from 'vitest';
import { schemaRegistry } from '../schema';
import { generateCode } from '../../../src/codegen/generator';
const code = generateCode(
schemaRegistry(`
interface Text { value: pb<1,string>; }
interface Nested { texts: pb_repeated<1,Text>; }
`),
);
const { encode, decode, encodeNested, decodeNested } = new Function(`${code}
return {
encode: protobuf_encode_Text, decode: protobuf_decode_Text,
encodeNested: protobuf_encode_Nested, decodeNested: protobuf_decode_Nested,
};`)() as {
decode(bytes: Uint8Array): { value: string };
encode(value: { value: string }): Uint8Array;
encodeNested(value: { texts: { value: string }[] }): Uint8Array;
decodeNested(bytes: Uint8Array): { texts: { value: string }[] };
};
const encoder = new TextEncoder();
const decoder = new TextDecoder();
function wire(payload: Uint8Array): Uint8Array {
const prefix = [10];
let length = payload.length;
do {
prefix.push((length & 127) | (length >= 128 ? 128 : 0));
length >>>= 7;
} while (length);
return Uint8Array.from([...prefix, ...payload]);
}
describe('UTF-8 codecs', () => {
it('matches TextEncoder across native encoding boundaries, including unpaired surrogates', () => {
for (const length of [31, 32, 33, 42, 43, 63, 64, 65, 127, 128, 129, 4096]) {
for (const unit of ['a', 'é', '中文', '🚀', '\ud800', '\udc00', 'a\ud800']) {
const value = unit.repeat(length);
// Generic iterable equality repeats per-byte work on these large views.
deepStrictEqual(encode({ value }), wire(encoder.encode(value)));
}
}
});
it('preserves every ASCII byte across short and native decoding boundaries', () => {
for (const length of [
0, 1, 3, 4, 7, 8, 15, 16, 23, 24, 30, 31, 32, 33, 63, 64, 127, 128, 4096,
]) {
for (let byte = 0; byte < 128; byte++) {
const value = String.fromCharCode(byte).repeat(length);
deepStrictEqual(
encode({ value }),
length ? wire(encoder.encode(value)) : new Uint8Array(),
);
expect(decode(wire(encoder.encode(value)))).toEqual({ value });
}
}
});
it('matches TextDecoder for Unicode after ASCII prefixes and at slice boundaries', () => {
const suffixes = [
'\u0080',
'é',
'\u07ff',
'\u0800',
'中文',
'\ud7ff',
'\ue000',
'\uffff',
'🚀',
'\u{10000}',
'\u{10ffff}',
'\ufeff',
'\ufeffx\ufeff',
'\ud800',
'\udc00',
];
for (let length = 0; length <= 64; length++) {
for (const suffix of suffixes) {
const value = 'a'.repeat(length) + suffix;
const payload = encoder.encode(value);
const bytes = wire(payload);
const padded = Uint8Array.from([255, 255, ...bytes, 128, 128]);
expect(decode(padded.subarray(2, 2 + bytes.length)).value).toBe(
decoder.decode(payload),
);
}
}
});
it('matches replacement behavior for all one-byte and two-byte sequences', () => {
const payload = new Uint8Array(2);
for (let first = 0; first < 256; first++) {
payload[0] = first;
const one = payload.subarray(0, 1);
expect(decode(wire(one)).value).toBe(decoder.decode(one));
for (let second = 0; second < 256; second++) {
payload[1] = second;
expect(decode(wire(payload)).value).toBe(decoder.decode(payload));
}
}
});
it('matches TextDecoder across randomized valid Unicode strings', () => {
let state = 0x5a17cafe;
for (let sample = 0; sample < 5000; sample++) {
let value = 'a'.repeat(sample % 17);
for (let index = 0; index < sample % 21; index++) {
state = (Math.imul(state, 1664525) + 1013904223) >>> 0;
value += String.fromCodePoint(state % 0x110000);
}
const payload = encoder.encode(value);
expect(decode(wire(payload)).value).toBe(decoder.decode(payload));
}
});
it('matches TextDecoder for invalid and truncated multibyte sequences', () => {
const sequences = [
[0xe0, 0x80, 0x80],
[0xed, 0xa0, 0x80],
[0xf0, 0x80, 0x80, 0x80],
[0xf4, 0x90, 0x80, 0x80],
[0xf5, 0x80, 0x80, 0x80],
[0xff],
[0xc2],
[0xe1, 0x80],
[0xf1, 0x80, 0x80],
[0xe1, 65, 0x80],
[0xf1, 0x80, 65, 0x80],
[0xef, 0xbb, 0xbf, 0xff],
];
for (let length = 0; length <= 33; length++) {
for (const sequence of sequences) {
const payload = Uint8Array.from([...Array(length).fill(65), ...sequence]);
expect(decode(wire(payload)).value).toBe(decoder.decode(payload));
}
}
let state = 0x42abcdef;
for (let sample = 0; sample < 5000; sample++) {
const payload = Uint8Array.from({ length: sample % 65 }, () => {
state = (Math.imul(state, 1664525) + 1013904223) >>> 0;
return state >>> 24;
});
expect(decode(wire(payload)).value).toBe(decoder.decode(payload));
}
});
it('keeps nested and repeated strings independent', () => {
const texts = [
'',
'ascii',
'a'.repeat(31),
'a'.repeat(32),
'中文🚀',
'a\ufeffb',
'\ufeffx',
'中文🚀'.repeat(64),
].map((value) => ({ value }));
expect(decodeNested(encodeNested({ texts }))).toEqual({
texts: texts.map(({ value }) => ({ value: decoder.decode(encoder.encode(value)) })),
});
// A truncated UTF-8 sequence must not consume bytes from the following field.
const first = wire(Uint8Array.from([0xf0, 0x9f]));
const second = wire(encoder.encode('next'));
const nested = Uint8Array.from([10, first.length, ...first, 10, second.length, ...second]);
expect(decodeNested(nested)).toEqual({ texts: [{ value: '�' }, { value: 'next' }] });
});
it('measures UTF-8 correctly when a code point crosses a scratch-buffer boundary', () => {
for (const length of [4093, 4094, 4095, 4096, 4097, 8191]) {
for (const suffix of ['é', '中', '🚀', '\ud800', '\udc00', '\ud800x']) {
const value = 'a'.repeat(length) + suffix + '中文🚀'.repeat(4000);
deepStrictEqual(encode({ value }), wire(encoder.encode(value)));
}
}
});
it('preserves replacement and BOM behavior after long malformed inputs', () => {
for (const length of [127, 128, 129, 255, 4095, 4096, 4097]) {
for (const sequence of [
[0xff],
[0xe0, 0x80, 0x80],
[0xed, 0xa0, 0x80],
[0xf0, 0x9f],
[0xef, 0xbb, 0xbf],
]) {
for (const prefix of [[], [0xef, 0xbb, 0xbf]]) {
const payload = Uint8Array.from([
...prefix,
...Array(length).fill(65),
...sequence,
]);
expect(decode(wire(payload)).value).toBe(decoder.decode(payload));
const valid = encoder.encode('中文🚀'.repeat(64));
expect(decode(wire(valid)).value).toBe(decoder.decode(valid));
}
}
}
});
});
@@ -0,0 +1,202 @@
import { describe, it, expect } from 'vitest';
import { schemaCodecs as codecs } from '../schema';
describe('generated wire format', () => {
it('matches TextEncoder at UTF-8 length-prefix boundaries', () => {
const codec = codecs<{ value: string }>('interface Msg {value:pb<1,string>;}');
for (const length of [127, 128, 16383, 16384]) {
for (const value of ['a'.repeat(length), '界🚀\ud800'.repeat(length)]) {
const payload = new TextEncoder().encode(value);
const prefix = [10];
let remaining = payload.length;
do {
prefix.push((remaining % 128) | (remaining >= 128 ? 128 : 0));
remaining = Math.floor(remaining / 128);
} while (remaining);
const expected = new Uint8Array(prefix.length + payload.length);
expected.set(prefix);
expected.set(payload, prefix.length);
expect(codec.encode({ value })).toEqual(expected);
expect(codec.decode(expected)).toEqual({
value: new TextDecoder().decode(payload),
});
}
}
}, 30_000);
it('decodes floating-point, fixed-width and byte fields from unaligned subarrays', () => {
const codec = codecs<{ f: number; d: number; fixed: bigint; data: Uint8Array }>(
`interface Msg {f:pb<1,float>;d:pb<2,double>;fixed:pb<3,fixed_64>;data:pb<4,bytes>;}`,
);
const input = {
f: Math.fround(1 / 3),
d: -Number.MAX_VALUE,
fixed: (1n << 64n) - 1n,
data: new Uint8Array([0, 1, 255]),
};
const wire = codec.encode(input);
for (const padding of [1, 3, 7, 31]) {
const buffer = new Uint8Array(wire.length + padding + 5).fill(255);
buffer.set(wire, padding);
expect(codec.decode(buffer.subarray(padding, padding + wire.length))).toEqual(input);
}
});
it('matches an independent bigint varint reference at every byte boundary', () => {
const codec = codecs<{ value: bigint }>('interface Msg { value: pb<1, uint_64>; }');
const values = new Set<bigint>([0n, 1n, (1n << 64n) - 1n]);
for (let bit = 1n; bit < 64n; bit++) {
values.add((1n << bit) - 1n);
values.add(1n << bit);
values.add((1n << bit) + 1n);
}
for (const value of values) {
let rest = value;
const bytes = [];
do {
bytes.push(Number(rest & 127n) | (rest > 127n ? 128 : 0));
rest >>= 7n;
} while (rest);
const wire = new Uint8Array([8, ...bytes]);
expect(codec.encode({ value })).toEqual(value === 0n ? new Uint8Array() : wire);
expect(codec.decode(wire)).toEqual({ value });
}
});
it('decodes nonzero booleans whose low 32 bits are zero', () => {
const codec = codecs<{ value: boolean }>('interface Msg { value: pb<1, bool>; }');
expect(codec.decode(new Uint8Array([8, 128, 128, 128, 128, 16]))).toEqual({ value: true });
});
const uint32 = codecs<{ value: number }>('interface Msg { value: pb<1, uint_32>; }');
const varints: [number, number[]][] = [
[1, [0x01]],
[127, [0x7f]],
[128, [0x80, 0x01]],
[300, [0xac, 0x02]],
[16384, [0x80, 0x80, 0x01]],
[0xffffffff, [0xff, 0xff, 0xff, 0xff, 0x0f]],
];
it.each(varints)('encodes uint32 %i using the expected wire bytes', (value, bytes) => {
expect([...uint32.encode({ value })]).toEqual([0x08, ...bytes]);
});
it.each(varints)('decodes uint32 %i from independent wire bytes', (value, bytes) => {
expect(uint32.decode(new Uint8Array([0x08, ...bytes]))).toEqual({ value });
});
it('omits scalar zero and decodes an explicit zero', () => {
expect(uint32.encode({ value: 0 })).toEqual(new Uint8Array());
expect(uint32.decode(new Uint8Array([8, 0]))).toEqual({ value: 0 });
});
it.each(['hello', '', '你好', '🚀', '\ud800', '\udc00'])('encodes UTF-8 %j', (value) => {
const codec = codecs<{ value: string }>('interface Msg { value: pb<1, string>; }');
const payload = new TextEncoder().encode(value);
const bytes = new Uint8Array([0x0a, payload.length, ...payload]);
expect(codec.encode({ value })).toEqual(value === '' ? new Uint8Array() : bytes);
expect(codec.decode(bytes)).toEqual({ value: new TextDecoder().decode(payload) });
});
it('encodes bytes with a length prefix and copies the decoded data', () => {
const codec = codecs<{ value: Uint8Array }>('interface Msg { value: pb<1, bytes>; }');
const bytes = new Uint8Array([10, 3, 1, 128, 255]);
const value = new Uint8Array([1, 128, 255]);
expect(codec.encode({ value })).toEqual(bytes);
const result = codec.decode(bytes);
bytes[2] = 0;
expect(result.value).toEqual(value);
});
it.each([
['varint', [0x10, 0xac, 0x02]],
['64-bit', [0x11, 1, 2, 3, 4, 5, 6, 7, 8]],
['length-delimited', [0x12, 3, 0x41, 0x42, 0x43]],
['32-bit', [0x15, 1, 2, 3, 4]],
])('skips unknown %s fields and resumes decoding', (_name, bytes) => {
expect(uint32.decode(new Uint8Array([...(bytes as number[]), 8, 7]))).toEqual({ value: 7 });
});
const packedCases: [string, (number | bigint | boolean)[], number[]][] = [
['uint_32', [1, 300], [1, 0xac, 2]],
['int_32', [1, -2], [1, 0xfe, ...Array(8).fill(0xff), 1]],
['sint_32', [-1, 1], [1, 2]],
['bool', [true, false], [1, 0]],
['uint_64', [1n, 300n], [1, 0xac, 2]],
['int_64', [1n, -2n], [1, 0xfe, ...Array(8).fill(0xff), 1]],
['sint_64', [-1n, 1n], [1, 2]],
['fixed_32', [0xffffffff, 1], [255, 255, 255, 255, 1, 0, 0, 0]],
['sfixed_32', [-1, 1], [255, 255, 255, 255, 1, 0, 0, 0]],
['fixed_64', [1n], [1, 0, 0, 0, 0, 0, 0, 0]],
['sfixed_64', [-1n], Array(8).fill(255)],
['float', [1], [0, 0, 128, 63]],
['double', [1], [0, 0, 0, 0, 0, 0, 240, 63]],
];
it.each(packedCases)(
'decodes packed, unpacked and split %s arrays',
(type, values, payload) => {
const codec = codecs<{ values: typeof values }>(
`interface Msg { values: pb_repeated<1, ${type}>; }`,
);
const packed = [10, payload.length, ...payload];
const unpacked = codec.encode({ values });
expect(codec.decode(new Uint8Array(packed))).toEqual({ values });
expect(codec.decode(new Uint8Array([...packed, ...unpacked, ...packed]))).toEqual({
values: [...values, ...values, ...values],
});
expect(codec.decode(new Uint8Array([10, 0]))).toEqual({ values: [] });
},
);
const malformedCodec = codecs(`
interface Inner { value: pb<1, uint_32>; }
interface Msg {
value: pb<1, uint_32>;
big: pb<2, uint_64>;
text: pb<3, string>;
data: pb<4, bytes>;
float: pb<5, float>;
double: pb<6, double>;
inner: pb<7, Inner>;
values: pb_repeated<8, uint_32>;
fixed: pb_repeated<9, fixed_32>;
}
`);
const invalidCases: [string, number[]][] = [
['zero tag', [0]],
['invalid wire type', [15]],
['truncated tag', [128]],
['overflowing tag', [255, 255, 255, 255, 31]],
['missing varint', [8]],
['unterminated varint', [8, 128]],
['overflowing varint', [8, ...Array(10).fill(255)]],
['truncated bigint', [16, 128]],
['overflowing bigint', [16, ...Array(9).fill(255), 2]],
['missing string length', [26]],
['truncated string', [26, 2, 65]],
['overflowing length', [26, 255, 255, 255, 255, 31]],
['truncated bytes', [34, 2, 1]],
['truncated float', [45, 0]],
['truncated double', [49, 0]],
['nested message exceeds parent', [58, 5, 8, 1]],
['nested varint exceeds nested boundary', [58, 2, 8, 128, 8, 7]],
['packed varint exceeds packed boundary', [66, 1, 128, 8, 7]],
['packed fixed32 has partial value', [74, 3, 1, 2, 3, 8, 7]],
['unknown truncated varint', [80, 128]],
['unknown truncated fixed64', [81, 0]],
['unknown truncated bytes', [82, 5, 1]],
['unknown truncated fixed32', [85, 0]],
];
it.each(invalidCases)('rejects %s', (_name, bytes) => {
expect(() => malformedCodec.decode(new Uint8Array(bytes))).toThrow(/protobuf/);
});
});
+24
View File
@@ -0,0 +1,24 @@
import ts from 'typescript';
import { ProtobufCompiler } from '../../src/compiler/compiler';
import { packageName } from '../../src/package';
export const imports = `import { protobuf_encode, protobuf_decode } from '${packageName}';`;
export function run(body: string): any {
const compiler = new ProtobufCompiler();
const source = imports + '\n' + body;
const transformed = compiler.compile(source, 'test-entry.ts') ?? source;
const js = ts.transpileModule(transformed, {
compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.CommonJS },
}).outputText;
const exports = {};
new Function('exports', 'require', js)(exports, () => ({
protobuf_encode() {
throw new Error('Uncompiled encode');
},
protobuf_decode() {
throw new Error('Uncompiled decode');
},
}));
return exports;
}
@@ -0,0 +1,332 @@
import { bundle } from '../project';
import { describe, expect, it } from 'vitest';
import { run, imports } from '../compile';
const schema = `interface Msg { value: pb<1,uint_32>; }
function encode<T>(value:T) { return protobuf_encode<T>(value); }`;
describe('generic callbacks', () => {
it.each([
'const callbacks=[encode];',
'const api={send:encode};',
'[{value:7}].map(<T>(value:T)=>protobuf_encode<T>(value));',
'[{value:7}].map((encode));',
])('rejects callback values without enough schema context: %s', (expression) => {
expect(() => run(schema + expression)).toThrow(/Cannot infer callback schema/);
});
it('supports callbacks in default parameters and codec calls in parameter initializers', () => {
expect(
run(`${schema}
function apply<T>(value:T, callback:(value:T)=>Uint8Array=protobuf_encode) {return callback(value);}
function defaults<T>(value:T, bytes=protobuf_encode<T>(value)) {return bytes;}
export const result=[[...apply<Msg>({value:7})],[...defaults<Msg>({value:8})]];
export const lengths=[apply.length, defaults.length];
`),
).toEqual({
result: [
[8, 7],
[8, 8],
],
lengths: [1, 1],
});
});
it('supports destructured polymorphic callback parameters', () => {
expect(
run(`${schema}
interface Api {encode:<T>(value:T)=>Uint8Array;}
function use<T>({encode:fn}:Api,value:T) {return fn<T>(value);}
export const result=[...use<Msg>({encode},{value:7})];
`).result,
).toEqual([8, 7]);
});
it('propagates object callback signatures through function parameters', () => {
expect(
run(`${schema}
interface Api {encode:<T>(value:T)=>Uint8Array;}
function use<T>(api:Api,value:T) {return api.encode<T>(value);}
export const result=[...use<Msg>({encode},{value:7})];
`).result,
).toEqual([8, 7]);
});
it('adapts typed assignments and callbacks stored in arrays', () => {
expect(
run(`${schema}
type Encoder=(value:Msg)=>Uint8Array;
let callback:Encoder;
callback=encode;
const callbacks:Encoder[]=[encode,callback];
export const result=callbacks.map(fn=>[...fn({value:7})]);
`).result,
).toEqual([
[8, 7],
[8, 7],
]);
});
it('reports opaque polymorphic consumers at build time', () => {
expect(() =>
run(`${schema}
declare function external(fn:<T>(value:T)=>Uint8Array):void;
external(encode);
`),
).toThrow(/receiving implementation/);
});
it('supports class arrow fields as callbacks with lexical this', () => {
expect(
run(`${schema}
class Codec<T> { calls=0; encode=(value:T)=>{this.calls++;return protobuf_encode<T>(value);}; }
const codec=new Codec<Msg>();
export const result=[{value:7}].map(codec.encode).map(bytes=>[...bytes]);
export const calls=codec.calls;
`),
).toEqual({ result: [[8, 7]], calls: 1 });
});
it('preserves spread evaluation, method receivers and rest arguments', () => {
expect(
run(`${schema}
const events:string[]=[];
const api={calls:0,send<T>(value:T,...labels:string[]){this.calls++;events.push(...labels);return protobuf_encode<T>(value);}};
function args():[Msg,string,string] {events.push('args');return [{value:7},'a','b'];}
export const result=[...api.send<Msg>(...args())];
export const calls=api.calls;
export {events};
`),
).toEqual({ result: [8, 7], calls: 1, events: ['args', 'a', 'b'] });
});
it('analyzes shared higher order functions across multiple build entries', async () => {
expect(
await bundle(
{
'apply.ts':
'export function apply<T>(value:T, callback:<U>(value:U)=>Uint8Array) { return callback<T>(value); }',
'plain.ts': `${imports} import {apply} from './apply'; interface Msg {value:pb<1,uint_32>;}
function encode<T>(value:T) {return protobuf_encode<T>(value);}
export const result=[...apply<Msg>({value:7},encode)];`,
'boxed.ts': `${imports} import {apply} from './apply'; interface Msg {value:pb<1,uint_32>;} interface Box<T> {value:pb<1,T>;}
function encode<T>(value:T) {return protobuf_encode<Box<T>>({value});}
export const result=[...apply<Msg>({value:8},encode)];`,
},
['plain.ts', 'boxed.ts'],
),
).toEqual({ plain: { result: [8, 7] }, boxed: { result: [10, 2, 8, 8] } });
});
it('preserves this parameters, bound generic methods and class arguments', () => {
expect(
run(`${schema}
interface Pair<T,U> {first:pb<1,T>;second:pb<2,U>;}
class Codec<T> {
calls=0;
encode<U>(this:Codec<T>, value:Pair<T,U>) { this.calls++; return protobuf_encode<Pair<T,U>>(value); }
}
const codec = new Codec<uint_32>();
const callback = codec.encode<string>.bind(codec);
export const result = [...callback({first:7,second:'x'})];
export const calls = codec.calls;
`),
).toEqual({ result: [8, 7, 18, 1, 120], calls: 1 });
});
it('supports returned polymorphic functions and typed object callbacks', () => {
expect(
run(`${schema}
type Encoder = <T>(value:T)=>Uint8Array;
function factory(): Encoder { return encode; }
const callback = factory();
const api: {encode:Encoder} = {encode};
export const result = [[...callback<Msg>({value:7})], [...api.encode<Msg>({value:8})], [...factory()<Msg>({value:9})]];
`).result,
).toEqual([
[8, 7],
[8, 8],
[8, 9],
]);
});
it('supports nested generic callback aliases', () => {
expect(
run(`${schema}
type Callback<T> = (value:T)=>Uint8Array;
type Alias<T> = Callback<T>;
const callback: Alias<Msg> = encode;
export const result = [...callback({value:7})];
`).result,
).toEqual([8, 7]);
});
it('passes all required codecs when callback implementations differ', () => {
expect(
run(`${schema}
interface Box<T> {value:pb<1,T>;}
function boxed<T>(value:T) { return protobuf_encode<Box<T>>({value}); }
function apply<T>(value:T, callback:<U>(value:U)=>Uint8Array) { return callback<T>(value); }
export const result = [[...apply<Msg>({value:7}, encode)], [...apply<Msg>({value:8}, boxed)]];
`).result,
).toEqual([
[8, 7],
[10, 2, 8, 8],
]);
});
it('supports a callback implementation accepting fewer parameters than its signature', () => {
expect(
run(`${schema}
function apply(callback:<T>(value:T, index:number)=>Uint8Array) { return callback<Msg>({value:7}, 42); }
export const result = [...apply(encode)];
`).result,
).toEqual([8, 7]);
});
it('preserves default parameter evaluation and callback length', () => {
expect(
run(`${schema}
let defaults=0;
function callback<T>(value:T, flag=++defaults) { return flag ? protobuf_encode<T>(value) : new Uint8Array(); }
const cb = callback<Msg>;
export const result = [...cb({value:7})];
export const length = cb.length;
export {defaults};
`),
).toEqual({ result: [8, 7], length: 1, defaults: 1 });
});
it('propagates nested schema substitutions through callback chains', () => {
expect(
run(`${schema}
interface Box<T> {value:pb<1,T>;}
function nested<T>(value:Box<T>) { return protobuf_encode<Box<T>>(value); }
function apply<T>(value:Box<T>, fn:<U>(value:Box<U>)=>Uint8Array) { return fn<T>(value); }
export const result = [...apply<Msg>({value:{value:7}}, nested)];
`).result,
).toEqual([10, 2, 8, 7]);
});
it('preserves async callback bodies', async () => {
const result = run(`${schema}
async function send<T>(value:T) { await Promise.resolve(); return protobuf_encode<T>(value); }
const values: Msg[] = [{value:7},{value:8}];
export const result = Promise.all(values.map(send));
`);
expect((await result.result).map((bytes: Uint8Array) => [...bytes])).toEqual([
[8, 7],
[8, 8],
]);
});
it('materializes instantiation expressions in variables, returns and arrays', () => {
expect(
run(`${schema}
const callback = encode<Msg>;
function factory() { return encode<Msg>; }
const values = [encode<Msg>, factory(), callback];
export const result = values.map(fn => [...fn({value:7})]);
`).result,
).toEqual([
[8, 7],
[8, 7],
[8, 7],
]);
});
it('infers array callbacks and preserves side effects and rest arguments', () => {
expect(
run(`${schema}
const values: Msg[] = [{value:7}, {value:8}];
const calls: number[] = [];
function callback<T>(value:T, ...rest: unknown[]) { calls.push(rest[0] as number); return protobuf_encode<T>(value); }
export const result = values.map(callback).map(bytes => [...bytes]);
export { calls };
`),
).toEqual({
result: [
[8, 7],
[8, 8],
],
calls: [0, 1],
});
});
it('adapts runtime function values directly', () => {
expect(
run(`${schema}
const values: Msg[] = [{value:7}];
const encodeMsg = protobuf_encode<Msg>;
const decodeMsg: (bytes: Uint8Array) => Msg = protobuf_decode;
export const result = [decodeMsg(encodeMsg(values[0])), ...values.map(protobuf_encode).map(decodeMsg)];
`).result,
).toEqual([{ value: 7 }, { value: 7 }]);
});
it('passes codecs through a generic higher order function', () => {
expect(
run(`${schema}
function apply<T>(value:T, callback:(value:T)=>Uint8Array) { return callback(value); }
export const result = [...apply<Msg>({value:7}, encode)];
`).result,
).toEqual([8, 7]);
});
it('supports independently generic callbacks and transitive forwarding', () => {
expect(
run(`${schema}
type Encoder = <T>(value:T)=>Uint8Array;
function invoke<T>(value:T, callback:Encoder) { return callback<T>(value); }
function forward<T>(value:T, callback:Encoder) { return invoke<T>(value, callback); }
export const result = [...forward<Msg>({value:7}, encode)];
`).result,
).toEqual([8, 7]);
});
it('keeps distinct schemas when one generic callback is reused', () => {
expect(
run(`${schema}
interface Text { value: pb<1,string>; }
function both(callback:<T>(value:T)=>Uint8Array) {
return [[...callback<Msg>({value:7})], [...callback<Text>({value:'x'})]];
}
export const result = both(encode);
`).result,
).toEqual([
[8, 7],
[10, 1, 120],
]);
});
it('captures codecs in returned callbacks and contextual aliases', () => {
expect(
run(`${schema}
type Encoder<T> = (value:T)=>Uint8Array;
function factory<T>(): Encoder<T> { return encode; }
const callback: Encoder<Msg> = encode;
export const result = [[...callback({value:7})], [...factory<Msg>()({value:8})]];
`).result,
).toEqual([
[8, 7],
[8, 8],
]);
});
it('supports generic inline callbacks and preserves native thisArg', () => {
expect(
run(`${schema}
const values: Msg[] = [{value:7}];
const state = { calls:0 };
function callback<T>(value:T) { this.calls++; return protobuf_encode<T>(value); }
export const a = values.map(callback, state).map(bytes => [...bytes]);
export const b = values.map(<T>(value:T) => protobuf_encode<T>(value)).map(bytes => [...bytes]);
export const calls = state.calls;
`),
).toEqual({ a: [[8, 7]], b: [[8, 7]], calls: 1 });
});
it('handles imported higher order functions and callback aliases', async () => {
expect(
await bundle({
'encoder.ts': `${imports} export function encode<T>(value:T) { return protobuf_encode<T>(value); }`,
'apply.ts': `export function apply<T>(value:T, fn:<U>(value:U)=>Uint8Array) { return fn<T>(value); }`,
'entry.ts': `import {encode as callback} from './encoder'; import {apply} from './apply'; interface Msg {value:pb<1,uint_32>;} export const result = [...apply<Msg>({value:7}, callback)];`,
}),
).toEqual({ result: [8, 7] });
});
});
@@ -0,0 +1,196 @@
import { bundle } from '../project';
import { describe, expect, it } from 'vitest';
import { run, imports } from '../compile';
import { ProtobufCompiler } from '../../../src/compiler/compiler';
const box = 'interface Box<T> {value:pb<1,T>;}';
const codec = `${box}
class Codec<T> {
encode(value:Box<T>) {return protobuf_encode<Box<T>>(value);}
decode(bytes:Uint8Array) {return protobuf_decode<Box<T>>(bytes);}
}`;
describe('generic class instances', () => {
it('isolates uint32 and string instances across alternating calls', () => {
const result = run(`${codec}
const numbers=new Codec<uint_32>(); const strings=new Codec<string>();
export const result=Array.from({length:32},(_,i)=>[numbers.decode(numbers.encode({value:i})),strings.decode(strings.encode({value:'v'+i}))]);
export const sharedMethod=numbers.encode===strings.encode;
export const keys=Object.keys(numbers);
`);
expect(result.result).toEqual(
Array.from({ length: 32 }, (_, i) => [{ value: i }, { value: 'v' + i }]),
);
expect(result.sharedMethod).toBe(true);
expect(result.keys).toEqual([]);
});
it('emits one codec per concrete type for many instances', () => {
const declarations = Array.from(
{ length: 32 },
(_, i) => `const n${i}=new Codec<uint_32>(); const s${i}=new Codec<string>();`,
).join('\n');
const output = new ProtobufCompiler().compile(imports + codec + declarations, 'reuse.ts')!;
expect(output.match(/function protobuf_encode_Box__uint_32\(/g)).toHaveLength(1);
expect(output.match(/function protobuf_decode_Box__uint_32\(/g)).toHaveLength(1);
expect(output.match(/function protobuf_encode_Box__string\(/g)).toHaveLength(1);
expect(output.match(/function protobuf_decode_Box__string\(/g)).toHaveLength(1);
});
it('alternates method type arguments on one instance', () => {
expect(
run(`${box}
class Codec {roundtrip<T>(value:Box<T>) {return protobuf_decode<Box<T>>(protobuf_encode<Box<T>>(value));}}
const instance=new Codec();
export const result=[instance.roundtrip<uint_32>({value:7}),instance.roundtrip<string>({value:'x'}),instance.roundtrip<uint_32>({value:8})];
`).result,
).toEqual([{ value: 7 }, { value: 'x' }, { value: 8 }]);
});
it('forwards outer generics through factories returning class instances', () => {
expect(
run(`${codec}
function create<T>() {return new Codec<T>();}
const n=create<uint_32>(); const s=create<string>();
export const result=[n.decode(n.encode({value:7})),s.decode(s.encode({value:'x'}))];
`).result,
).toEqual([{ value: 7 }, { value: 'x' }]);
});
it('combines factory class arguments and independent method arguments', () => {
expect(
run(`interface Pair<T,U> {first:pb<1,T>;second:pb<2,U>;}
class Codec<T> {roundtrip<U>(value:Pair<T,U>) {return protobuf_decode<Pair<T,U>>(protobuf_encode<Pair<T,U>>(value));}}
function create<T>():Codec<T> {return new Codec<T>();}
const n=create<uint_32>(); const s=create<string>(); const alias=n;
export const result=[n.roundtrip<string>({first:7,second:'x'}),s.roundtrip<uint_32>({first:'y',second:8}),alias.roundtrip<uint_32>({first:9,second:10}),create<string>().roundtrip<string>({first:'a',second:'b'})];
`).result,
).toEqual([
{ first: 7, second: 'x' },
{ first: 'y', second: 8 },
{ first: 9, second: 10 },
{ first: 'a', second: 'b' },
]);
});
it('infers constructor generics from annotated arguments', () => {
expect(
run(`${box}
class Codec<T> {constructor(readonly value:Box<T>) {} encode(){return protobuf_encode<Box<T>>(this.value);}}
const n:Box<uint_32>={value:7}; const s:Box<string>={value:'x'};
const numbers=new Codec(n); const strings=new Codec(s);
export const result=[[...numbers.encode()],[...strings.encode()]];
`).result,
).toEqual([
[8, 7],
[10, 1, 120],
]);
});
it('infers factories and inherited constructors with renamed parameters', () => {
expect(
run(`${box}
class Codec<T> {constructor(readonly value:Box<T>) {} encode(){return protobuf_encode<Box<T>>(this.value);}}
class Middle<V> extends Codec<V> {}
class Derived<U> extends Middle<U> {}
function create<W>(value:Box<W>) {return new Derived(value);}
const n:Box<uint_32>={value:7}; const s:Box<string>={value:'x'};
const numbers=create(n); const strings=create(s);
export const result=[[...numbers.encode()],[...strings.encode()]];
`).result,
).toEqual([
[8, 7],
[10, 1, 120],
]);
});
it('specializes unannotated factories, aliases and default class arguments', () => {
expect(
run(`interface Pair<T,U> {first:pb<1,T>;second:pb<2,U>;}
class Codec<T=string> {roundtrip<U>(value:Pair<T,U>) {return protobuf_decode<Pair<T,U>>(protobuf_encode<Pair<T,U>>(value));}}
const create=<T>()=>new Codec<T>();
type Alias<T> = Codec<T>;
function use<T>(instance:Alias<T>, first:T) {return instance.roundtrip<string>({first,second:'ok'});}
export const result=[create<uint_32>().roundtrip<string>({first:7,second:'x'}),new Codec().roundtrip<uint_32>({first:'y',second:8}),use<string>(create<string>(),'z')];
`).result,
).toEqual([
{ first: 7, second: 'x' },
{ first: 'y', second: 8 },
{ first: 'z', second: 'ok' },
]);
});
it('retains class bindings in generic callback adapters', () => {
expect(
run(`interface Pair<T,U> {first:pb<1,T>;second:pb<2,U>;}
class Codec<T> {roundtrip<U>(value:Pair<T,U>) {return protobuf_decode<Pair<T,U>>(protobuf_encode<Pair<T,U>>(value));}}
function create<T>():Codec<T> {return new Codec<T>();}
const numbers=create<uint_32>(); const strings=create<string>();
const n=numbers.roundtrip<string>.bind(numbers); const s=strings.roundtrip<uint_32>.bind(strings);
export const result=[n({first:7,second:'x'}),s({first:'y',second:8})];
`).result,
).toEqual([
{ first: 7, second: 'x' },
{ first: 'y', second: 8 },
]);
});
it('separates specializations through inherited constructors', () => {
expect(
run(`${codec}
class Derived<T> extends Codec<T> {}
const n=new Derived<uint_32>(); const s=new Derived<string>();
export const result=[n.decode(n.encode({value:7})),s.decode(s.encode({value:'x'}))];
`).result,
).toEqual([{ value: 7 }, { value: 'x' }]);
});
it('keeps cross-file factory instances specialized', async () => {
expect(
await bundle({
'codec.ts': `${imports} ${codec} export function create<T>() {return new Codec<T>();}`,
'entry.ts': `import {create} from './codec'; const n=create<uint_32>();const s=create<string>(); export const result=[n.decode(n.encode({value:7})),s.decode(s.encode({value:'x'}))];`,
}),
).toEqual({ result: [{ value: 7 }, { value: 'x' }] });
});
it('resolves method defaults from the class type argument', () => {
expect(
run(`interface Pair<T,U> {first:pb<1,T>;second:pb<2,U>;}
class Codec<T> {roundtrip<U=T>(value:Pair<T,U>) {return protobuf_decode<Pair<T,U>>(protobuf_encode<Pair<T,U>>(value));}}
function create<T>():Codec<T> {return new Codec<T>();}
export const result=[create<uint_32>().roundtrip({first:7,second:8}),create<string>().roundtrip({first:'x',second:'y'})];
`).result,
).toEqual([
{ first: 7, second: 8 },
{ first: 'x', second: 'y' },
]);
});
it('specializes inherited static generic methods', () => {
expect(
run(`${box}
class Base {static roundtrip<T>(value:Box<T>) {return protobuf_decode<Box<T>>(protobuf_encode<Box<T>>(value));}}
class Codec extends Base {}
export const result=[Codec.roundtrip<uint_32>({value:7}),Codec.roundtrip<string>({value:'x'})];
`).result,
).toEqual([{ value: 7 }, { value: 'x' }]);
});
it('combines cross-file class and method generics', async () => {
expect(
await bundle({
'codec.ts': `${imports} export interface Pair<T,U> {first:pb<1,T>;second:pb<2,U>;}
class Codec<T> {roundtrip<U>(value:Pair<T,U>) {return protobuf_decode<Pair<T,U>>(protobuf_encode<Pair<T,U>>(value));}}
export const create=<T>()=>new Codec<T>();`,
'entry.ts': `import {create} from './codec'; const n=create<uint_32>(); const s=create<string>();
export const result=[n.roundtrip<string>({first:7,second:'x'}),s.roundtrip<uint_32>({first:'y',second:8})];`,
}),
).toEqual({
result: [
{ first: 7, second: 'x' },
{ first: 'y', second: 8 },
],
});
});
});
@@ -0,0 +1,424 @@
import { bundle } from '../project';
import { packageName } from '../../../src/package';
import { describe, expect, it } from 'vitest';
import { ProtobufCompiler } from '../../../src/compiler/compiler';
import { run, imports } from '../compile';
describe('semantic compiler', () => {
it('invalidates preloaded schema types when the transform receives different source', () => {
const compiler = new ProtobufCompiler();
const source = `${imports} interface Msg {value:pb<1,uint_32>;} protobuf_encode<Msg>({value:7});`;
compiler.context.source('preloaded.ts', source);
compiler.prepare(['preloaded.ts']);
expect(compiler.compile(source.replace('pb<1,', 'pb<2,'), 'preloaded.ts')).toContain(
'buf[offset++] = 16;',
);
});
it('preserves async generic function bodies and return values', async () => {
const result = run(`interface Msg { value: pb<1,uint_32>; }
async function encode<T>(value:T) { await Promise.resolve(); return protobuf_encode<T>(value); }
export const result = encode<Msg>({value:7});
`);
expect([...(await result.result)]).toEqual([8, 7]);
});
it('supports namespace markers and namespace runtime functions', () => {
expect(
run(`
import * as p from '${packageName}';
interface Msg { value: p.pb<1, p.uint_32>; }
export const result = [...p.protobuf_encode<Msg>({value: 7})];
`).result,
).toEqual([8, 7]);
});
it('supports default generic classes and re-exported runtime bindings', async () => {
expect(
await bundle({
'runtime.ts': `export { protobuf_encode as encode } from '${packageName}';`,
'codec.ts': `import { encode } from './runtime'; export default class Codec<T> { encode(value:T) { return encode<T>(value); } }`,
'entry.ts': `import Codec from './codec'; interface Msg { value: pb<1,uint_32>; } const codec = new Codec<Msg>(); export const result = [...codec.encode({value:7})];`,
}),
).toEqual({ result: [8, 7] });
});
it('resolves generic parameters containing dollar signs without text substitution', () => {
expect(
run(`interface Box<$T> { value: pb<1,$T>; }
function encode<$T>(value:Box<$T>) { return protobuf_encode<Box<$T>>(value); }
export const result = [...encode<string>({value:'x'})];
`).result,
).toEqual([10, 1, 120]);
});
it('preserves directive prologues', () => {
const code = `'use client';\n${imports}\ninterface Msg { value: pb<1,uint_32>; } protobuf_encode<Msg>({value:7});`;
expect(new ProtobufCompiler().compile(code, 'directive.ts')).toMatch(/^'use client';/);
});
it('does not mix anonymous fields instantiated with different generic arguments', () => {
expect(
run(`
interface Box<T> { nested: pb<1, { value: pb<1, T>; }>; }
export const a = protobuf_decode<Box<uint_32>>(protobuf_encode<Box<uint_32>>({ nested: { value: 7 } }));
export const b = protobuf_decode<Box<string>>(protobuf_encode<Box<string>>({ nested: { value: 'x' } }));
`),
).toEqual({ a: { nested: { value: 7 } }, b: { nested: { value: 'x' } } });
});
it('evaluates conditional schema types structurally', () => {
expect(
run(`
interface Base { id: pb<1, uint_32>; }
interface Child extends Base { text: pb<2, string>; }
type Chosen<T> = T extends Base ? Child : never;
export const result = protobuf_decode<Chosen<Child>>(protobuf_encode<Chosen<Child>>({ id: 7, text: 'x' }));
`).result,
).toEqual({ id: 7, text: 'x' });
});
it('supports destructured generic object methods and aliases', () => {
expect(
run(`
interface Msg { value: pb<1, uint_32>; }
const api = { encode<T>(value: T) { return protobuf_encode<T>(value); } };
const { encode: send } = api;
export const result = [...send<Msg>({ value: 7 })];
`).result,
).toEqual([8, 7]);
});
it('rejects generic callbacks without a concrete adapter', () => {
expect(() =>
run(`
interface Msg { value: pb<1, uint_32>; }
function encode<T>(value: T) { return protobuf_encode<T>(value); }
[{ value: 7 }].map(encode);
`),
).toThrow(/Cannot infer callback schema/);
});
it('rejects detached instance codec methods', () => {
expect(() =>
run(`interface Msg { value: pb<1,uint_32>; }
class Codec<T> { encode(value:T) { return protobuf_encode<T>(value); } }
const codec = new Codec<Msg>(); const encode = codec.encode;
`),
).toThrow(/receiver/);
});
it('rejects class field calls that run before codec initialization', () => {
expect(() => run(`class Codec<T> { value = protobuf_encode<T>({} as T); }`)).toThrow(
/field initializers/,
);
});
it('keeps function length stable after generic codec injection', () => {
expect(
run(`
interface Msg { value: pb<1, uint_32>; }
function encode<T>(value: T) { return protobuf_encode<T>(value); }
export const length = encode.length;
export const result = [...encode<Msg>({ value: 7 })];
`),
).toEqual({ length: 1, result: [8, 7] });
});
it('reuses exact transform results and clears changed-source caches', () => {
const compiler = new ProtobufCompiler();
const source = `${imports} interface Msg { value: pb<1, uint_32>; } protobuf_encode<Msg>({value: 7});`;
const first = compiler.compile(source, 'cache.ts');
expect(compiler.compile(source, 'cache.ts')).toBe(first);
const second = compiler.compile(source.replace('pb<1,', 'pb<2,'), 'cache.ts');
expect(second).toContain('buf[offset++] = 16;');
expect(second).not.toEqual(first);
});
it('resolves tsconfig path aliases', async () => {
expect(
await bundle({
'tsconfig.json': JSON.stringify({
compilerOptions: { baseUrl: '.', paths: { '@schema/*': ['schema/*'] } },
}),
'schema/msg.ts': 'export interface Msg { value: pb<1, uint_32>; }',
'entry.ts': `${imports} import type { Msg } from '@schema/msg'; export const result = [...protobuf_encode<Msg>({value:7})];`,
}),
).toEqual({ result: [8, 7] });
});
it('supports recursive generic aliases and nested generic inference', () => {
expect(
run(`
type Node<T> = { value: pb<1, T>; next?: pb<2, Node<T>>; };
function encode<T>(value: Node<T>) { return protobuf_encode<Node<T>>(value); }
const value: Node<string> = { value: 'a', next: { value: 'b' } };
export const result = protobuf_decode<Node<string>>(encode(value));
`).result,
).toEqual({ value: 'a', next: { value: 'b', next: null } });
});
it('supports inferred generic forwarding and inherited generic methods', () => {
expect(
run(`
interface Pair<T, U> { a: pb<1, T>; b: pb<2, U>; }
class Base<T> { encode<U>(v: Pair<T, U>) { return protobuf_encode<Pair<T, U>>(v); } }
class Derived<T> extends Base<T> {}
const codec = new Derived<uint_32>();
export const result = [...codec.encode<string>({ a: 7, b: 'x' })];
`).result,
).toEqual([8, 7, 18, 1, 120]);
});
it('keeps generated helper names and user property strings separate', () => {
expect(
run(`
const __td = 'user decoder';
const protobuf_encode_Msg = 'user encoder';
interface Msg { '__td': pb<1, string>; }
export const result = protobuf_decode<Msg>(protobuf_encode<Msg>({ __td: 'ok' }));
export const untouched = [__td, protobuf_encode_Msg];
`),
).toEqual({ result: { __td: 'ok' }, untouched: ['user decoder', 'user encoder'] });
});
it('shares codecs across distinct instantiations without mixing schemas', () => {
expect(
run(`
interface A { value: pb<1, uint_32>; }
interface B { value: pb<2, string>; }
const encode = <T>(value: T) => protobuf_encode<T>(value);
export const result = [[...encode<A>({ value: 7 })], [...encode<B>({ value: 'x' })]];
`).result,
).toEqual([
[8, 7],
[18, 1, 120],
]);
});
it('supports generic field-number aliases and preserves mapped optionality', () => {
expect(
run(`
type Field<N extends number, T> = pb_optional<N, T>;
interface Msg { value?: Field<3, uint_32>; }
type Copy<T> = { [K in keyof T]: T[K] };
export const result = [...protobuf_encode<Copy<Msg>>({ value: 0 })];
`).result,
).toEqual([24, 0]);
});
it('forwards codecs through implicit and explicit generic derived constructors', () => {
expect(
run(`
interface Msg { value: pb<1, uint_32>; }
class Base<T> {
constructor(public label = 'base') {}
encode(value: T) { return protobuf_encode<T>(value); }
}
class Middle<T> extends Base<T> {}
class Derived<T> extends Middle<T> {
constructor(label: string) { super(label); }
}
const codec = new Derived<Msg>('ok');
export const result = [...codec.encode({ value: 7 })];
export const label = codec.label;
`),
).toEqual({ result: [8, 7], label: 'ok' });
});
it('forwards concrete codecs through a non-generic derived class', () => {
expect(
run(`
interface Msg { value: pb<1, uint_32>; }
class Base<T> { encode(value: T) { return protobuf_encode<T>(value); } }
class Codec extends Base<Msg> {}
export const result = [...new Codec().encode({ value: 7 })];
`).result,
).toEqual([8, 7]);
});
it('preserves explicitly present optional default values and quoted field names', () => {
const result = run(`
interface Msg { 'a-b'?: pb_optional<1, uint_32>; flag?: pb_optional<2, bool>; text?: pb_optional<3, string>; big?: pb_optional<4, uint_64>; '__proto__'?: pb<5, string>; }
const input = { 'a-b': 0, flag: false, text: '', big: 0n, ['__proto__']: 'safe' };
export const result = protobuf_decode<Msg>(protobuf_encode<Msg>(input));
`).result;
expect(result).toEqual({ 'a-b': 0, flag: false, text: '', big: 0n, ['__proto__']: 'safe' });
expect(Object.getPrototypeOf(result)).toBe(Object.prototype);
});
it('preserves imported generic wrappers and transitive aliases across .js imports and barrels', async () => {
const result = await bundle({
'types.ts': `export interface Box<T> { value: pb<1, T>; }`,
'codec.ts': `${imports} import type { Box as B } from './types.js';
let count = 0;
export function wrap<T>(value: B<T>) { count++; return protobuf_encode<B<T>>(value); }
export function calls() { return count; }`,
'barrel.ts': `export { wrap as encode, calls } from './codec.js'; export type { Box as Shape } from './types.js';`,
'entry.ts': `${imports} import { encode, calls } from './barrel.js'; import type { Shape } from './barrel.js';
export const result = protobuf_decode<Shape<string>>(encode<string>({ value: 'ok' }));
export const count = calls();`,
});
expect(result).toEqual({ result: { value: 'ok' }, count: 1 });
});
it('keeps same-named schemas from different files distinct', async () => {
const result = await bundle({
'a.ts': `export interface Msg { value: pb<1, uint_32>; }`,
'b.ts': `export interface Msg { value: pb<2, string>; }`,
'entry.ts': `${imports} import type { Msg as A } from './a'; import type * as b from './b';
export const a = [...protobuf_encode<A>({ value: 7 })];
export const b = [...protobuf_encode<b.Msg>({ value: 'x' })];`,
});
expect(result).toEqual({ a: [8, 7], b: [18, 1, 120] });
});
it('supports generic arrow functions and rest parameters without reordering user arguments', () => {
expect(
run(`
interface Msg { value: pb<1, uint_32>; }
const encode = <T>(value: T, ...suffix: number[]) => [...protobuf_encode<T>(value), ...suffix];
export const result = encode<Msg>({ value: 7 }, 9, 10);
`).result,
).toEqual([8, 7, 9, 10]);
});
it('does not rewrite shadowed protobuf identifiers or unrelated object methods', () => {
expect(
run(`
interface Msg { value: pb<1, uint_32>; }
function custom(protobuf_encode: <T>(v: T) => number) { return protobuf_encode<Msg>({ value: 7 }); }
const unrelated = { encode<T>(value: T) { return value; } };
export const result = custom(() => 99) + unrelated.encode<number>(1);
`).result,
).toBe(100);
});
it('preserves generic wrapper branches, argument positions, closures and side effects', () => {
const result = run(`
interface Msg { value: pb<1, uint_32>; }
let count = 0;
function wrap<I, T>(ignored: I, data: T, enabled = true) {
count++;
if (!enabled) return new Uint8Array([99]);
return protobuf_encode<T>(data);
}
export const a = [...wrap<string, Msg>('unused', { value: 7 })];
export const b = [...wrap<string, Msg>('unused', { value: 8 }, false)];
export const calls = count;
`);
expect(result).toEqual({ a: [8, 7], b: [99], calls: 2 });
});
it('propagates concrete codecs through multiple generic functions and nested closures', () => {
const result = run(`
interface Box<T> { value: pb<1, T>; }
function inner<T>(value: Box<T>) { return protobuf_encode<Box<T>>(value); }
function outer<T>(value: Box<T>) {
function forward() { return inner<T>(value); }
return forward();
}
export const result = protobuf_decode<Box<string>>(outer<string>({ value: 'ok' }));
`);
expect(result.result).toEqual({ value: 'ok' });
});
it('resolves aliases, generic defaults, inheritance, intersections and indexed access', () => {
const result = run(`
type Scalar = uint_32;
interface Base<T = Scalar> { id: pb<1, T>; }
interface Extra { title: pb<2, string>; }
interface Child extends Base { title: Extra['title']; }
type Alias<T> = T;
type Shape = Alias<Pick<Child & Extra, 'id' | 'title'>>;
export const result = protobuf_decode<Shape>(protobuf_encode<Shape>({ id: 7, title: 'x' }));
`);
expect(result.result).toEqual({ id: 7, title: 'x' });
});
it('supports mapped and conditional generic types', () => {
const result = run(`
interface Msg { id: pb<1, uint_32>; text: pb<2, string>; }
type Copy<T> = { [K in keyof T]: T[K] };
type Choice<T> = T extends string ? Copy<Msg> : never;
export const result = protobuf_decode<Choice<string>>(protobuf_encode<Choice<string>>({ id: 5, text: 'ok' }));
`);
expect(result.result).toEqual({ id: 5, text: 'ok' });
});
it('passes class type parameters through constructor and instance methods', () => {
const result = run(`
interface Msg { value: pb<1, uint_32>; }
class Codec<T> {
constructor(public label = 'default') {}
encode(data: T) { return protobuf_encode<T>(data); }
decode(bytes: Uint8Array) { return protobuf_decode<T>(bytes); }
}
const a = new Codec<Msg>();
const b = new Codec<Msg>('named');
export const result = a.decode(b.encode({ value: 9 }));
export const labels = [a.label, b.label];
export const keys = Object.keys(a);
`);
expect(result).toEqual({
result: { value: 9 },
labels: ['default', 'named'],
keys: ['label'],
});
});
it('supports generic class methods with class and method type arguments', () => {
const result = run(`
interface Pair<T, U> { first: pb<1, T>; second: pb<2, U>; }
class Codec<T> {
encode<U>(value: Pair<T, U>) { return protobuf_encode<Pair<T, U>>(value); }
}
const codec = new Codec<uint_32>();
export const result = protobuf_decode<Pair<uint_32, string>>(codec.encode<string>({ first: 7, second: 'x' }));
`);
expect(result.result).toEqual({ first: 7, second: 'x' });
});
it('supports recursive message schemas and nullable links', () => {
const result = run(`
interface Node { value: pb<1, uint_32>; next?: pb<2, Node | undefined>; }
export const result = protobuf_decode<Node>(protobuf_encode<Node>({ value: 1, next: { value: 2 } }));
`);
expect(result.result).toEqual({ value: 1, next: { value: 2, next: null } });
});
it('infers a generic argument from an annotated variable', () => {
const result = run(`
interface Msg { value: pb<1, uint_32>; }
const data: Msg = { value: 7 };
function wrap<T>(value: T) { return protobuf_encode<T>(value); }
export const result = [...wrap(data)];
`);
expect(result.result).toEqual([8, 7]);
});
it('does not inspect unused unsupported schema declarations', () => {
expect(
run(`
interface Unused { value: pb<1, MissingType>; }
interface Msg { value: pb<1, uint_32>; }
export const result = [...protobuf_encode<Msg>({ value: 7 })];
`).result,
).toEqual([8, 7]);
});
it.each(['0', '-1', '1.5', '19000', '536870912'])(
'rejects invalid field number %s at compile time',
(number) => {
expect(() =>
run(
`interface Msg { value: pb<${number}, uint_32>; } protobuf_encode<Msg>({ value: 7 });`,
),
).toThrow(packageName);
},
);
it('rejects duplicate field numbers', () => {
expect(() =>
run('interface Msg { a: pb<1, uint_32>; b: pb<1, string>; } protobuf_encode<Msg>({});'),
).toThrow(/Duplicate/);
});
});
@@ -0,0 +1,201 @@
import { describe, expect, it } from 'vitest';
import { run, imports } from '../compile';
import { bundle } from '../project';
import { packageName } from '../../../src/package';
const schema =
'interface Box<T> {value:pb<1,T>;} interface Pair<T,U> {first:pb<1,T>;second:pb<2,U>;}';
const codec = `${schema} class Codec<T> {
roundtrip<U>(value:Pair<T,U>) {return protobuf_decode<Pair<T,U>>(protobuf_encode<Pair<T,U>>(value));}
}`;
describe('generic composition', () => {
it('propagates two specializations across a 64-function call chain', () => {
const functions = Array.from(
{ length: 64 },
(_, i) =>
`function f${i}<T>(value:Box<T>) {return ${i === 63 ? 'protobuf_encode<Box<T>>(value)' : `f${i + 1}<T>(value)`};}`,
).join('\n');
expect(
run(
`${schema} ${functions} export const result=[[...f0<uint_32>({value:7})],[...f0<string>({value:'x'})]];`,
).result,
).toEqual([
[8, 7],
[10, 1, 120],
]);
});
it('composes deferred mapped types, promises, classes and polymorphic callbacks', () => {
expect(
run(`
interface Msg {id:pb<1,uint_32>;text:pb<2,string>;ignore:pb<3,bool>;}
type Payload<T> = Pick<Awaited<T>,'id'|'text'>;
interface Envelope<T> {value:pb<1,Payload<T>>;}
function encode<T>(value:Envelope<T>) {return protobuf_encode<Envelope<T>>(value);}
function apply<T>(value:Envelope<T>,fn:<U>(value:Envelope<U>)=>Uint8Array) {return fn<T>(value);}
class Codec<T> {encode(value:Envelope<T>){return apply<T>(value,encode);}}
const codec=new Codec<Promise<Msg>>();
export const result=protobuf_decode<Envelope<Msg>>(codec.encode({value:{id:7,text:'x'}}));
`).result,
).toEqual({ value: { id: 7, text: 'x' } });
});
it('retains outer type arguments across nested local factories', () => {
expect(
run(`${codec}
function outer<T>() {function inner() {return new Codec<T>();} return inner();}
const n=outer<uint_32>(); const s=outer<string>();
export const result=[n.roundtrip<string>({first:7,second:'x'}),s.roundtrip<uint_32>({first:'y',second:8})];
`).result,
).toEqual([
{ first: 7, second: 'x' },
{ first: 'y', second: 8 },
]);
});
it('resolves instances through parameter properties and getters', () => {
expect(
run(`${codec}
class Holder<T> {
constructor(readonly codec:Codec<T>) {}
get current():Codec<T> {return this.codec;}
}
const holder=new Holder<string>(new Codec<string>());
export const result=holder.current.roundtrip<uint_32>({first:'x',second:7});
`).result,
).toEqual({ first: 'x', second: 7 });
});
it('resolves static element access without changing this', () => {
expect(
run(`${codec}
const c=new Codec<uint_32>();
export const result=c['roundtrip']<string>({first:7,second:'x'});
`).result,
).toEqual({ first: 7, second: 'x' });
});
it('supports overloaded generic methods and constructors', () => {
expect(
run(`${schema}
class Codec<T> {
value:Box<T>;
constructor(value:Box<T>);
constructor(value:Box<T>, label:string);
constructor(value:Box<T>, public label='default') {this.value=value;}
encode() {return protobuf_encode<Box<T>>(this.value);}
roundtrip<U>(value:Box<U>):Box<U>;
roundtrip<U>(value:Box<U>):Box<U> {return protobuf_decode<Box<U>>(protobuf_encode<Box<U>>(value));}
}
const n:Box<uint_32>={value:7}; const c=new Codec(n);
export const result=[[...c.encode()],c.roundtrip<string>({value:'x'}),c.label];
`).result,
).toEqual([[8, 7], { value: 'x' }, 'default']);
});
it('forwards shadowed generic names through deeply nested callbacks', () => {
expect(
run(`${schema}
function roundtrip<T>(value:Box<T>) {return protobuf_decode<Box<T>>(protobuf_encode<Box<T>>(value));}
function outer<T>(first:Box<T>, fn:<T>(value:Box<T>)=>Box<T>) {
function middle<U>(next:Box<U>) {
function inner<T>(value:Box<T>) {return fn<T>(value);}
return [inner<T>(first),inner<U>(next)];
}
return middle<string>({value:'x'});
}
export const result=outer<uint_32>({value:7},roundtrip);
`).result,
).toEqual([{ value: 7 }, { value: 'x' }]);
});
it('converges through mutually recursive generic functions', () => {
expect(
run(`${schema}
function a<T>(value:Box<T>, depth:number):Box<T> {return depth ? b<T>(value,depth-1) : protobuf_decode<Box<T>>(protobuf_encode<Box<T>>(value));}
function b<U>(value:Box<U>, depth:number):Box<U> {return a<U>(value,depth);}
export const result=[a<uint_32>({value:7},32),a<string>({value:'x'},33)];
`).result,
).toEqual([{ value: 7 }, { value: 'x' }]);
});
it('keeps class factory specializations through cyclic re-export modules', async () => {
expect(
await bundle({
'codec.ts': `${imports} ${codec} export const create=<T>()=>new Codec<T>();`,
'first.ts': `export * from './second'; export {create} from './codec';`,
'second.ts': `export * from './first';`,
'entry.ts': `import {create} from './second'; export const result=create<string>().roundtrip<uint_32>({first:'x',second:7});`,
}),
).toEqual({ result: { first: 'x', second: 7 } });
});
it('recognizes parenthesized runtime aliases and namespace element access', () => {
expect(
run(`${schema} import * as runtime from '${packageName}';
const encode=protobuf_encode; const alias=encode;
export const result=[
[...(alias)<Box<uint_32>>({value:7})],
runtime['protobuf_decode']<Box<string>>(runtime['protobuf_encode']<Box<string>>({value:'x'}))
];
`).result,
).toEqual([[8, 7], { value: 'x' }]);
});
it('adapts typed callbacks selected by static element access', () => {
expect(
run(`${codec}
const c=new Codec<uint_32>();
const callback:(value:Pair<uint_32,string>)=>Pair<uint_32,string> = c['roundtrip'].bind(c);
export const result=callback({first:7,second:'x'});
`).result,
).toEqual({ first: 7, second: 'x' });
});
it('captures outer generics in a local class returned from a factory', () => {
expect(
run(`${schema}
function create<T>() {
class Local {roundtrip<U>(value:Pair<T,U>) {return protobuf_decode<Pair<T,U>>(protobuf_encode<Pair<T,U>>(value));}}
return new Local();
}
export const result=[create<uint_32>().roundtrip<string>({first:7,second:'x'}),create<string>().roundtrip<uint_32>({first:'y',second:8})];
`).result,
).toEqual([
{ first: 7, second: 'x' },
{ first: 'y', second: 8 },
]);
});
it('resolves namespaced base schemas and separates imported type and value names', async () => {
expect(
await bundle({
'types.ts': `export interface Base<T> {value:pb<1,T>;} export const Base=123;`,
'entry.ts': `${imports} import type * as types from './types';
interface Msg<T> extends types.Base<T> {text:pb<2,string>;}
export const result=protobuf_decode<Msg<uint_32>>(protobuf_encode<Msg<uint_32>>({value:7,text:'x'}));`,
}),
).toEqual({ result: { value: 7, text: 'x' } });
});
it('keeps valid recursive references through inherited generic fields', () => {
expect(
run(`
interface Base<T> {next:pb<1,Link<T>|null>;}
interface Link<T> extends Base<T> {value:pb<2,T>;}
const value:Link<string>={value:'a',next:{value:'b',next:null}};
export const result=protobuf_decode<Link<string>>(protobuf_encode<Link<string>>(value));
`).result,
).toEqual({ value: 'a', next: { value: 'b', next: null } });
});
it('resolves imported type and value names independently of declaration order', async () => {
expect(
await bundle({
'types.ts': `export const Msg=42; export interface Msg {value:pb<1,uint_32>;}`,
'barrel.ts': `export {Msg} from './types';`,
'entry.ts': `${imports} import {Msg} from './barrel'; export const result=[Msg,[...protobuf_encode<Msg>({value:7})]];`,
}),
).toEqual({ result: [42, [8, 7]] });
});
});
@@ -0,0 +1,114 @@
import { describe, expect, it } from 'vitest';
import { ProtobufCompiler } from '../../../src/compiler/compiler';
import { imports, run } from '../compile';
import { withProject } from '../project';
import { join } from 'node:path';
import { writeFileSync } from 'node:fs';
describe('compiler failure recovery', () => {
it('rejects circular class inheritance without looping during member resolution', () => {
expect(() =>
run(
`class A<T> extends B<T> {} class B<U> extends A<U> {} new A<uint_32>().missing();`,
),
).toThrow(/Circular class inheritance/);
});
it('rejects circular schema inheritance without caching an empty message', () => {
expect(() =>
run(`interface A extends B {} interface B extends A {} protobuf_encode<A>({});`),
).toThrow(/Circular schema inheritance/);
});
it('does not cache a partially validated schema after failure', () => {
const compiler = new ProtobufCompiler();
const source = `${imports} interface Msg {a:pb<1,uint_32>;b:pb<1,string>;} protobuf_encode<Msg>({a:7,b:'x'});`;
for (let i = 0; i < 3; i++) {
expect(() => compiler.compile(source, 'invalid.ts')).toThrow(
/Duplicate protobuf field number/,
);
}
expect(compiler.compile(source.replace('b:pb<1,', 'b:pb<2,'), 'invalid.ts')).toContain(
'protobuf_encode_Msg',
);
});
it('reports expanding recursive schema aliases before overflowing the stack', () => {
expect(() =>
run(
`type Grow<T>={value:pb<1,T>;next?:pb<2,Grow<Grow<T>>>;}; protobuf_encode<Grow<uint_32>>({value:7});`,
),
).toThrow(/128 levels|does not converge/);
});
it('reports circular aliases with a source location', () => {
expect(() => run(`type A=B; type B=A; protobuf_encode<A>({});`)).toThrow(
/test-entry.ts:\d+:\d+: Circular type alias/,
);
});
it('replaces specialized code when the same module repeatedly changes wire types', () => {
const compiler = new ProtobufCompiler();
for (let i = 0; i < 32; i++) {
const type = i % 2 ? 'string' : 'uint_32';
const other = i % 2 ? 'uint_32' : 'string';
const source = `${imports} interface Box<T> {value:pb<1,T>;} class Codec<T> {encode(value:Box<T>) {return protobuf_encode<Box<T>>(value);}} new Codec<${type}>();`;
const output = compiler.compile(source, 'changing.ts');
expect(output).toContain(`function protobuf_encode_Box__${type}(`);
expect(output).not.toContain(`function protobuf_encode_Box__${other}(`);
}
});
it('rejects ambiguous factory specializations with a build diagnostic', () => {
expect(() =>
run(`interface Box<T> {value:pb<1,T>;}
class Codec<T> {encode<U>(value:Box<T>) {return protobuf_encode<Box<T>>(value);}}
function create(flag:boolean) {if(flag) return new Codec<uint_32>();return new Codec<string>();}
create(true).encode<bool>({value:7});
`),
).toThrow(/Receiver branches must resolve to the same specialization/);
});
it('reconstructs all entry requirements after a failed compilation', async () => {
const shared =
'export function apply<T>(value:T, fn:<U>(value:U)=>Uint8Array) {return fn<T>(value);}';
await withProject(
{
'shared.ts': shared,
'a.ts': `${imports} import {apply} from './shared'; interface Msg {value:pb<1,uint_32>;} function encode<T>(value:T) {return protobuf_encode<T>(value);} apply<Msg>({value:7},encode);`,
'b.ts': `${imports} import {apply} from './shared'; interface Msg {value:pb<1,uint_32>;} interface Box<T> {value:pb<1,T>;} function encode<T>(value:T) {return protobuf_encode<Box<T>>({value});} apply<Msg>({value:8},encode);`,
},
(directory) => {
const compiler = new ProtobufCompiler();
compiler.prepare([join(directory, 'a.ts'), join(directory, 'b.ts')]);
const baseline = compiler.compile(shared, join(directory, 'shared.ts'));
expect(() =>
compiler.compile(
`${imports} protobuf_encode<Missing>({});`,
join(directory, 'bad.ts'),
),
).toThrow(/Unresolved type/);
expect(compiler.compile(shared, join(directory, 'shared.ts'))).toBe(baseline);
},
);
});
it('recovers when entry preparation fails before a corrected source is saved', async () => {
const source = `${imports} interface Msg {value:pb<1,uint_32>;} function encode<T>(value:T) {return protobuf_encode<T>(value);} encode({value:7});`;
await withProject({ 'entry.ts': source }, (directory) => {
const compiler = new ProtobufCompiler();
const file = join(directory, 'entry.ts');
for (let i = 0; i < 2; i++) {
expect(() => compiler.prepare([file])).toThrow(/Cannot infer T/);
}
const corrected = source.replace('encode({', 'encode<Msg>({');
writeFileSync(file, corrected);
compiler.prepare([file]);
expect(compiler.compile(corrected, file)).toContain('protobuf_encode_Msg');
});
});
});
@@ -0,0 +1,75 @@
import { describe, expect, it } from 'vitest';
import { run } from '../compile';
const schema = 'interface Msg {value:pb<1,uint_32>;}';
describe('call evaluation', () => {
it('preserves optional receiver short circuiting and single evaluation', () => {
expect(
run(`${schema}
const events:string[]=[];
class Codec {encode<T>(value:T) {events.push('encode');return protobuf_encode<T>(value);}}
function get(present:boolean):Codec|undefined {events.push('get');return present ? new Codec() : undefined;}
function argument():Msg {events.push('argument');return {value:7};}
export const absent=get(false)?.encode<Msg>(argument());
export const result=[...get(true)?.encode<Msg>(argument())!];
export {events};
`),
).toEqual({
absent: undefined,
result: [8, 7],
events: ['get', 'get', 'argument', 'encode'],
});
});
it('preserves getter receivers, multiple spreads and trailing commas', () => {
expect(
run(`${schema}
const events:string[]=[];
const codec={encode<T>(value:T,...labels:string[]) {events.push(...labels);return protobuf_encode<T>(value);}};
const holder={get current(){events.push('get');return codec;}};
function values():[Msg] {events.push('values');return [{value:7}];}
function labels():string[] {events.push('labels');return ['a','b'];}
export const result=[...holder.current.encode<Msg>(...values(),...labels(),)];
export {events};
`),
).toEqual({ result: [8, 7], events: ['get', 'values', 'labels', 'a', 'b'] });
});
it('evaluates constructor defaults before the body and preserves rest arguments', () => {
expect(
run(`${schema}
const events:string[]=[];
class Codec<T> {
constructor(public value:T, public bytes=protobuf_encode<T>(value), ...labels:string[]) {events.push(...labels);}
decode() {return protobuf_decode<T>(this.bytes);}
}
function args():[Msg,undefined,string,string] {events.push('args');return [{value:7},undefined,'a','b'];}
const c=new Codec<Msg>(...args());
export const result=c.decode(); export {events};
export const length=Codec.length;
`),
).toEqual({ result: { value: 7 }, events: ['args', 'a', 'b'], length: 1 });
});
it('preserves default codec calls in generic callback implementations', () => {
expect(
run(`${schema}
function encode<T>(value:T, bytes=protobuf_encode<T>(value)) {return bytes;}
function apply<T>(value:T, fn:<U>(value:U, bytes?:Uint8Array)=>Uint8Array) {return fn<T>(value);}
export const result=[...apply<Msg>({value:7},encode)];
`).result,
).toEqual([8, 7]);
});
it('supports generator and async generator bodies', async () => {
const result = run(`${schema}
function* encode<T>(value:T) {yield protobuf_encode<T>(value);}
async function* later<T>(value:T) {yield protobuf_encode<T>(value);}
export const result=[...encode<Msg>({value:7})].map(bytes=>[...bytes]);
export const pending=later<Msg>({value:8}).next();
`);
expect(result.result).toEqual([[8, 7]]);
expect([...(await result.pending).value]).toEqual([8, 8]);
});
});
@@ -0,0 +1,54 @@
import { bundle } from '../project';
import { describe, expect, it } from 'vitest';
import { imports } from '../compile';
describe('module graph', () => {
it('resolves nested imported schemas and renamed exports', async () => {
expect(
await bundle({
'inner.ts': 'export interface Inner {value:pb<1,uint_32>;} ',
'outer.ts':
"import type {Inner} from './inner'; export interface Outer {inner:pb<1,Inner>;}",
'barrel.ts': "export {Outer as Message} from './outer';",
'entry.ts': `${imports} import type {Message as Msg} from './barrel'; export const result = protobuf_decode<Msg>(protobuf_encode<Msg>({inner:{value:7}}));`,
}),
).toEqual({ result: { inner: { value: 7 } } });
});
it('keeps wrapper control flow, object receivers and transitive generic aliases', async () => {
expect(
await bundle({
'schema.ts':
'export interface Box<T> {value:pb<1,T>;} export type Message<T> = Box<Box<T>>;',
'codec.ts': `${imports} import type {Message} from './schema';
export const api = { count:0, encode<T>(value:Message<T>, enabled=true) {
this.count++; if (!enabled) return new Uint8Array();
const inner = () => protobuf_encode<Message<T>>(value); return inner();
}};`,
'entry.ts': `${imports} import {api} from './codec'; import type {Message} from './schema';
export const result = protobuf_decode<Message<string>>(api.encode<string>({value:{value:'x'}}));
export const empty = api.encode<string>({value:{value:'x'}}, false).length;
export const count = api.count;`,
}),
).toEqual({ result: { value: { value: 'x' } }, empty: 0, count: 2 });
});
it('handles recursive types across modules', async () => {
expect(
await bundle({
'a.ts': "import type {B} from './b'; export interface A {value:pb<1,uint_32>; next?:pb<2,B>;}",
'b.ts': "import type {A} from './a'; export interface B {text:pb<1,string>; next?:pb<2,A>;}",
'entry.ts': `${imports} import type {A} from './a'; export const result = protobuf_decode<A>(protobuf_encode<A>({value:7,next:{text:'x'}}));`,
}),
).toEqual({ result: { value: 7, next: { text: 'x', next: null } } });
});
it('resolves nested generic functions with non-first schema parameters', async () => {
expect(
await bundle({
'codec.ts': `${imports} export function send<Context,T>(context:Context, value:T) { if (!context) throw new Error('context'); return protobuf_encode<T>(value); }`,
'entry.ts': `import {send} from './codec'; interface Msg {value:pb<1,uint_32>;} export const result = [...send<string,Msg>('ok',{value:7})];`,
}),
).toEqual({ result: [8, 7] });
});
});
@@ -0,0 +1,87 @@
import { describe, expect, it } from 'vitest';
import { run } from '../compile';
const schema = 'interface Msg {id:pb<1,uint_32>; text:pb<2,string>;}';
describe('schema type transformations', () => {
it('uses class parameter properties and ignores static metadata', () => {
expect(
run(`
class Msg<T> {static label='message';constructor(public value:pb<1,T>) {}}
export const result=protobuf_decode<Msg<uint_32>>(protobuf_encode<Msg<uint_32>>(new Msg<uint_32>(7)));
`).result,
).toEqual({ value: 7 });
});
it('captures function generics in local schemas without mixing instantiations', () => {
expect(
run(`
function encode<T>(value:T) { interface Local {value:pb<1,T>;} return protobuf_encode<Local>({value}); }
export const result=[[...encode<uint_32>(7)], [...encode<string>('x')]];
`).result,
).toEqual([
[8, 7],
[10, 1, 120],
]);
});
it('extracts schema types through infer, functions and promises', () => {
expect(
run(`${schema}
type Result<T> = T extends (...args:any[])=>infer R ? R : never;
type Unwrap<T> = T extends Promise<infer U> ? Unwrap<U> : T;
type Schema = Unwrap<Result<(id:number)=>Promise<Promise<Msg>>>>;
export const result = protobuf_decode<Schema>(protobuf_encode<Schema>({id:7,text:'x'}));
`).result,
).toEqual({ id: 7, text: 'x' });
});
it('supports Awaited, ReturnType, Parameters and type queries', () => {
expect(
run(`${schema}
function load(value:Msg): Promise<Msg> {return Promise.resolve(value);}
type Result = Awaited<ReturnType<typeof load>>;
type Input = Parameters<typeof load>[0];
export const result = protobuf_decode<Result>(protobuf_encode<Input>({id:7,text:'x'}));
`).result,
).toEqual({ id: 7, text: 'x' });
});
it('propagates deferred conditional transformations through generic functions', () => {
expect(
run(`${schema}
type Unwrap<T> = T extends Promise<infer U> ? U : T;
function encode<T>(value:Unwrap<T>) {return protobuf_encode<Unwrap<T>>(value);}
export const result = protobuf_decode<Msg>(encode<Promise<Msg>>({id:7,text:'x'}));
`).result,
).toEqual({ id: 7, text: 'x' });
});
it('remaps literal field names and omits never keys', () => {
expect(
run(`${schema}
type Rename<T> = {[K in keyof T as K extends 'text' ? never : \`field_\${Capitalize<K>}\`]:T[K]};
export const result = protobuf_decode<Rename<Msg>>(protobuf_encode<Rename<Msg>>({field_Id:7}));
`).result,
).toEqual({ field_Id: 7 });
});
it('selects explicit schemas with Extract and Exclude', () => {
expect(
run(`${schema}
type Schema = Extract<Msg|string,object>;
type Chosen = Exclude<Msg|string,string>;
export const result = protobuf_decode<Chosen>(protobuf_encode<Schema>({id:7,text:'x'}));
`).result,
).toEqual({ id: 7, text: 'x' });
});
it('supports constructor signatures and indexed tuple transformations', () => {
expect(
run(`${schema}
type Constructor = new(value:Msg)=>Msg;
type Instance = InstanceType<Constructor>;
type Input = ConstructorParameters<Constructor>[0];
export const result = protobuf_decode<Instance>(protobuf_encode<Input>({id:7,text:'x'}));
`).result,
).toEqual({ id: 7, text: 'x' });
});
});
@@ -0,0 +1,25 @@
import { expect, it, vi } from 'vitest';
vi.mock('../../../package.json', () => ({ name: '@test/renamed-codec' }));
import protobufPlugin from '../../../src/index';
import { ProtobufCompiler } from '../../../src/compiler/compiler';
import { protobuf_encode } from '../../../src/runtime';
it('uses the manifest name for imports, plugin identity and diagnostics', () => {
expect(protobufPlugin().name).toBe('@test/renamed-codec:vite');
expect(
new ProtobufCompiler().compile(
`import {protobuf_encode} from '@test/renamed-codec';
interface Msg {value:pb<1,uint_32>;} protobuf_encode<Msg>({value:7});`,
'renamed.ts',
),
).toContain('protobuf_encode_Msg({value:7})');
expect(() => protobuf_encode({})).toThrow('@test/renamed-codec Vite plugin');
expect(() =>
new ProtobufCompiler().compile(
`import {protobuf_encode} from '@test/renamed-codec'; protobuf_encode<Missing>({});`,
'error.ts',
),
).toThrow('[@test/renamed-codec]');
});
@@ -0,0 +1,108 @@
import { describe, expect, it } from 'vitest';
import protobufPlugin from '../../../src/index';
import { packageName } from '../../../src/package';
import { ProtobufCompiler } from '../../../src/compiler/compiler';
import { run, imports } from '../compile';
import { withProject } from '../project';
import { join } from 'node:path';
import { writeFileSync } from 'node:fs';
const schema = 'interface Msg { value: pb<1,uint_32>; }';
describe('plugin entry', () => {
it('watches type-only dependencies and rebuilds importers after schema changes', async () => {
await withProject(
{ 'message.ts': 'export interface Msg { value: pb<1, uint_32>; }' },
(directory) => {
const plugin = protobufPlugin();
const transformHook = plugin.transform!;
const transform =
typeof transformHook === 'function' ? transformHook : transformHook.handler;
const dependencies: string[] = [];
const context = { addWatchFile: (path: string) => dependencies.push(path) } as any;
const file = join(directory, 'entry.ts').replaceAll('\\', '/');
const message = join(directory, 'message.ts').replaceAll('\\', '/');
const source = `${imports} import type { Msg } from './message'; protobuf_encode<Msg>({value:7});`;
const initial = transform.call(context, source, file) as { code: string };
expect(initial.code).toContain('buf[offset++] = 8;');
expect(dependencies).toContain(message);
writeFileSync(message, 'export interface Msg { value: pb<2, uint_32>; }');
const updateHook = plugin.handleHotUpdate!;
const update = typeof updateHook === 'function' ? updateHook : updateHook.handler;
const module = { id: file };
expect(
update.call(
{} as any,
{
file: message,
server: {
moduleGraph: {
getModuleById: (id: string) =>
id === file ? module : undefined,
},
},
} as any,
),
).toEqual([module]);
const updated = transform.call(context, source, file) as { code: string };
expect(updated.code).toContain('buf[offset++] = 16;');
},
);
});
it('recognizes the package name from package.json', () => {
const plugin = protobufPlugin();
const hook = plugin.transform!;
const transform = typeof hook === 'function' ? hook : hook.handler;
const result = transform.call(
{} as any,
`import {protobuf_encode as encode} from '${packageName}'; ${schema} encode<Msg>({value:7});`,
'entry.ts',
) as { code: string };
expect(result.code).toContain('protobuf_encode_Msg({value:7})');
});
it('leaves unrelated modules and declaration files alone', () => {
const plugin = protobufPlugin();
const hook = plugin.transform!;
const transform = typeof hook === 'function' ? hook : hook.handler;
for (const file of ['entry.js', 'schema.d.ts', 'schema.d.mts']) {
expect(transform.call({} as any, schema, file)).toBeNull();
}
expect(new ProtobufCompiler().compile(schema, 'schema.ts')).toBeNull();
});
it('generates only codecs reachable from actual calls', () => {
const code = new ProtobufCompiler().compile(
`import {protobuf_encode} from '${packageName}'; ${schema}
interface Unused {value:pb<1,Missing>;}
protobuf_encode<Msg>({value:7});`,
'used.ts',
)!;
expect(code).not.toContain('function protobuf_decode_');
expect(code).not.toContain('function protobuf_encode_Unused');
expect(code).toContain('function protobuf_encode_Msg');
});
it('reports unresolved types instead of leaving runtime calls behind', () => {
expect(() => run('protobuf_encode<Missing>({});')).toThrow(/Unresolved type Missing/);
});
it('fails clearly when the build plugin is absent', async () => {
const runtime = await import('../../../src/runtime');
expect(Object.keys(runtime).sort()).toEqual([
'protobuf_decode',
'protobuf_encode',
'protobuf_getUnknownFieldMetadata',
'protobuf_getUnknownFields',
]);
expect(() => runtime.protobuf_encode({})).toThrow(
`not transformed by the ${packageName} Vite plugin`,
);
expect(() => runtime.protobuf_decode(new Uint8Array())).toThrow(
`not transformed by the ${packageName} Vite plugin`,
);
});
});
+79
View File
@@ -0,0 +1,79 @@
import { build } from 'vite';
import protobufPlugin from '../../src/index';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, dirname, posix } from 'node:path';
import { packageName } from '../../src/package';
export async function withProject<T>(
files: Record<string, string>,
action: (directory: string) => T | Promise<T>,
): Promise<T> {
const directory = mkdtempSync(join(tmpdir(), 'protobuf-compiler-'));
try {
for (const [name, text] of Object.entries(files)) {
const path = join(directory, name);
mkdirSync(dirname(path), { recursive: true });
writeFileSync(path, text);
}
return await action(directory);
} finally {
rmSync(directory, { recursive: true, force: true });
}
}
export function bundle(
files: Record<string, string>,
entry: string | string[] = 'entry.ts',
): Promise<any> {
return withProject(files, async (directory) => {
const output = await build({
configFile: false,
root: directory,
logLevel: 'silent',
plugins: [protobufPlugin()],
build: {
write: false,
minify: false,
target: 'es2022',
lib: {
entry: Array.isArray(entry)
? entry.map((file) => join(directory, file))
: join(directory, entry),
formats: ['cjs'],
},
rolldownOptions: { external: [packageName] },
},
});
const result = (Array.isArray(output) ? output[0] : output) as {
output: { code?: string; fileName: string; name: string; isEntry?: boolean }[];
};
const cache = new Map<string, Record<string, unknown>>();
const load = (name: string): Record<string, unknown> => {
if (cache.has(name)) {
return cache.get(name)!;
}
const chunk = result.output.find((chunk) => chunk.fileName === name);
if (!chunk?.code) {
throw new Error(`Unexpected runtime dependency: ${name}`);
}
const exports = {};
cache.set(name, exports);
new Function('exports', 'require', chunk.code)(exports, (specifier: string) =>
load(posix.normalize(posix.join(posix.dirname(name), specifier))),
);
return Object.fromEntries(Object.entries(exports));
};
const entries = result.output.filter((chunk) => chunk.isEntry);
return Array.isArray(entry)
? Object.fromEntries(entries.map((chunk) => [chunk.name, load(chunk.fileName)]))
: load(entries[0].fileName);
});
}
+40
View File
@@ -0,0 +1,40 @@
import ts from 'typescript';
import { SourceContext } from '../../src/source/context';
import { SchemaResolver } from '../../src/schema/resolver';
import { generateCode } from '../../src/codegen/generator';
export function schemaCodecs<T>(code: string, name = 'Msg') {
const generated = generateCode(schemaRegistry(code));
return new Function(
`${generated}\nreturn { encode: protobuf_encode_${name}, decode: protobuf_decode_${name} };`,
)() as {
encode(value: T): Uint8Array;
decode(bytes: Uint8Array): T;
};
}
export function schemaRegistry(code: string, file = 'schema.ts') {
const context = new SourceContext();
const source = context.source(file, code);
const resolver = new SchemaResolver(context);
const roots: string[] = [];
for (const statement of source.statements) {
if (ts.isInterfaceDeclaration(statement) && !statement.typeParameters?.length) {
roots.push(statement.name.text);
}
}
const expanded = context.source(
file,
code + roots.map((name, i) => `\ntype __TestRoot${i} = ${name};`).join(''),
);
for (const statement of expanded.statements) {
if (ts.isTypeAliasDeclaration(statement) && statement.name.text.startsWith('__TestRoot')) {
resolver.message(resolver.type(statement.type), statement);
}
}
return resolver.registry;
}
+20
View File
@@ -0,0 +1,20 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"types": ["node"],
"noEmit": true
},
"include": [
"src/**/*",
"test/__tests__/**/*",
"test/helpers/**/*",
"test/upstream/**/*",
"protobuf.d.ts"
],
"exclude": [
"node_modules",
"dist",
"test/fixtures"
]
}
+7
View File
@@ -0,0 +1,7 @@
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
include: ['test/**/*.test.ts'],
},
});