quickscript

1.Intro🔗

This post will just serve as me rambling about a project I'm working on, aka, trying to plan it and setting stuff up for when I actually implement it so I can reason through it before I realize I fucked up about 5 hours into writing an interpreter.

I want to start with some base ideas that I'm going to try to keep in mind when designing the internal structure of the language's interpreter.

Link to the interpreter source code: github.com/juliewoolie/quickscript

All data exists on the stack.

That tagline is a bit hyperbolic. I've since discovered that there are some things where its easier for me to pass by reference (arrays, strings, structs) so this mantra exists as a general guideline now.

No memory interaction

You should never be able to interact with raw pointers, random memory or anything like C allows you to. It's a scripting language, not C 2.0.

Type safety and basically compiled

quickscript is a statically typed language and my hope is a type system will allow me to make the most out of what I'm doing. Static typing means that the compiler basically knows where everything will be, how big it will be and how it'll be used.

No garbage collector

Memory handling should be done as it executes, the hope is that with a simple stack-memory-based language data will be freed as scopes are exited and so there's no requirement for automatic garbage collection.

Single threaded

I'm not managing that shit, at least not right now. This is called quickscript because it's intended to be a quickly (ish) made scripting language.

No nulls

This doesn't mean every value has to have a non-zero value, rather, there can't be anything which can be null, everything has to have a value, even if it's a zero value.

This also doesn't mean every array and string has to be allocated to an empty value, rather it means the user should never be able to tell something is null.

Calling .length on a null string or array? Return 0 instead of throwing a null reference error.

2.Language🔗

2.1.Notation🔗

References to other syntactic rules are in italic, literal values are in bold. An italic syntactic rule followed by a colon (:) defines the rule. Alternative definitions of the rule are defined on new lines. Optional parts are described with the subscript "opt". And parts which may repeat and appear several times will be described with the suffix "rep".

Indicates an syntax element enclosed with "()" which might contain an expression, or several expressions.

2.2.Lexical Elements🔗

token:

keyword

identifier

constant

string-literal

operator

2.2.1.Keywords🔗

Keyword tokens are divided into two separate categories: real and alias keywords.

The difference between alias keywords and real keywords, is that while the alias keywords are still reserved keywords, their internal token value is mapped to the value that they are an alias for. For example, ulong is an alias for uint64.

keyword:

real-keyword

type-keyword

alias-keyword

real-keyword: one of:

• if
• else
• break
• continue
• return
• while
• for
• struct
• module
• import
• export
• do
• const
• assert
• native

type-keyword: one of:

• void
• uint8
• bool
• int8
• uint16
• int16
• uint32
• int32
• uint64
• int64
• float32
• float64
• string

alias-keyword: one of:

• boolean
• byte
• ubyte
• char
• uchar
• short
• ushort
• int
• uint
• long
• ulong
• float
• double

2.2.2.Identifiers🔗

identifier:

identifier-start identifier-continue_{opt rep}

identifier-start:

nondigit

XID_Start character (defined by Unicode)

identifier-continue:

digit

nondigit

XID_Continue character (defined by Unicode)

digit: one of:

0 1 2 3 4 5 6 7 8 9

nondigit: one of:

_ $ a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

2.2.3.Constants🔗

constant:

integer-constant

floating-point-constant

character-constant

predefined-constant

2.2.3.1.Integer Constants🔗

integer-constant:

digit-sequence

octal-constant

hexadecimal-constant

binary-constant

octal-constant:

0o octal-digit_rep

0O octal-digit_rep

octal-digit: one of:

0 1 2 3 4 5 6 7

hexadecimal-constant:

0x hexadecimal-digit_rep

0X hexadecimal-digit_rep

hexadecimal-digit: one of:

0 1 2 3 4 5 6 7 8 9 a b c d e f A B C D E F

binary-constant:

0b binary-digit_rep

0B binary-digit_rep

binary-digit: one of:

0 1

digit-sequence:

digit

digit-sequence underscore_opt digit

underscore:

_

2.2.3.2.Floating Point Constants🔗

floating-point-constant:

fractional-part exponent-part_opt

fractional-part:

digit-sequence

digit-sequence . digit-sequence

digit-sequence .

. digit-sequence

exponent-part:

e sign_opt digit-sequence

E sign_opt digit-sequence

sign: one of:

+ -

2.2.3.3.Character Constants🔗

character-constant:

' char-sequence '

char-sequence:

Any valid UTF-8 character except the single-quote, backslash or newline character

escape-sequence

escape-sequence:

simple-escape-sequence

hex-escape-sequence

simple-escape-sequence: one of:

\' \" \\ \t \r \n \T \R \N

hex-escape-sequence:

\u hexadecimal-digit_rep

\U hexadecimal-digit_rep

2.2.3.4.Predefined Constants🔗

predefined-constant:

true

false

2.2.4.String Literals🔗

string-literal:

" string-character_{opt rep} "

string-character:

Any valid UTF-8 character except the double-quote, backslash or newline character

escape-sequence

2.2.5.Operators🔗

operator: one of:

( ) { } [ ] : ; , . ... & &= && &&= | |= * *= / /= + += - -= ^ ^= % %= << <<= >> >>= >>> >>>= < <= > >= ++ -- == != = ! ~ => ** **= || ||= ?

2.2.6.Comments🔗

Except within a string or character literal, the characters // and # denote the beginning of a line comment. The comment consumes the rest of the source file's line.

Except within a string or character literal, the characters /* denote the beginning of a block comment. The comment consumes input until a matching */ sequence has been found.

2.3.Expressions🔗

2.3.1.Primary Expressions🔗

primary-expression:

constant

identifier

object-literal

array-literal

( expression )

2.3.1.1.Object Literals🔗

object-literal:

{ object-literal-property_{opt rep} }

object-literal-property:

identifier : expression

2.3.1.2.Array Literals🔗

array-literal:

[ array-literal-values ]

array-literal-values:

expression

array-literal-values , expression

2.3.2.Trail Expressions🔗

trail-expression:

primary-expression

trail-expression . identifier

trail-expression [ expression ]

trail-expression ( arguments-list_opt )

arguments-list:

expression

arguments-list , expression

2.3.3.Unary Expressions🔗

unary-expression:

trail-expression

unary-expression ++

unary-expression --

unary-operator unary-expression

unary-operator: one of:

++ -- + - ~ !

2.3.4.Power (POW) Expression🔗

power-expression

unary-expression

power-expression ** unary-expression

2.3.5.Multiplicative Expressions🔗

multiplicative-expression

power-expression

multiplicative-expression * power-expression

multiplicative-expression / power-expression

multiplicative-expression % power-expression

2.3.6.Additive Expressions🔗

additive-expression:

multiplicative-expression

additive-expression + multiplicative-expression

additive-expression - multiplicative-expression

2.3.7.Shift Expressions🔗

bitshift-expression:

additive-expression

bitshift-expression << additive-expression

bitshift-expression >> additive-expression

2.3.8.Relational Expressions🔗

relational-expression:

bitshift-expression

relational-expression < bitshift-expression

relational-expression <= bitshift-expression

relational-expression > bitshift-expression

relational-expression >= bitshift-expression

2.3.9.Equality Expressions🔗

equality-expression:

relational-expression

equality-expression == relational-expression

equality-expression != relational-expression

2.3.10.XOR Expressions🔗

xor-expression:

equality-expression

xor-expression ^ equality-expression

2.3.11.Bitwise AND Expressions🔗

bitwise-and-expression:

xor-expression

bitwise-and-expression & xor-expression

2.3.12.Bitwise OR Expressions🔗

bitwise-or-expression:

bitwise-and-expression

bitwise-or-expression | bitwise-and-expression

2.3.13.Logical AND Expressions🔗

logical-and-expression:

bitwise-or-expression

logical-and-expression && bitwise-or-expression

2.3.14.Logical OR Expressions🔗

logical-or-expression:

logical-and-expression

logical-or-expression || logical-and-expression

2.3.15.Conditional Expression🔗

conditional-expression:

logical-or-expression

logical-or-expression ? expression : conditional-expression

2.3.16.Assignment Expression🔗

assignment-expression:

conditional-expression

conditional-expression assignment-operator assignment-expression

assignment-operator: one of:

= &= &&= |= *= /= += -= ^= %= <<= >>= **= ||=

2.3.17.Loop Condition Expression🔗

loop-condition-expression:

( expression )

expression

Description

Loop condition expressions declare a while expression that may or may not be surrounded by parentheses. Loop condition expressions declared with parentheses cannot have any part of the expression outside of the parentheses

Constraints

1. Loop condition expressions must always result in a value that can be assigned in some way to a true/false value.

2.3.18.Top Level Expression🔗

expression:

assignment-expression

2.4.Type Expressions🔗

type-expression:

array-type-expression

2.4.1.Const Type Expressions🔗

const-type-expression:

type-keyword

alias-keyword

2.4.2.Type Name Expressions🔗

type-name-expression:

const-type-expression

identifier

2.4.3.Array Type Expressions🔗

array-type-expression:

type-name-expression

array-name-expression []

2.5.Declarations🔗

declaration:

function-declaration

struct-declaration

variable-declaration

declaration-modifiers:

declaration-modifier_rep

declaration-modifier: one of:

native export const

2.5.1.Function Declarations🔗

function-declaration:

declaration-modifiers_opt type-expression identifier ( function-argument-list_opt ) block-statement_opt

function-argument-list:

function-argument-declaration

function-argument-list , function-argument-declaration

function-argument-declaration:

type-expression identifier ..._opt

Description

Declares a function that will be accessible in the current scope.

Constraints

1. The name and parameter types must not match a function that already exists in the current scope.
2. Function argument lists can only ever have one variadic argument: the last one. Multiple variadic arguments or a variadic argument not being the last one are invalid.
3. Functions declared with the export keyword can only be declared in the global scope.
4. Functions declared with the native keyword must not have a function body.

2.5.2.Variable Declarations🔗

variable-declaration:

declaration-modifiers type-expression identifier

declaration-modifiers type-expression identifier = expression

Description

Declares a variable or a constant that will be accessible in the current scope.

Constraints

1. Must not have the name of an already accessible variable
2. If a value is specified for the declaration, the variable's resulting value must match the declared type.
3. If the variable is declared with const, then a value must be specified.
4. Variables declared with the native modifier cannot ever have a value. As the native keyword declares a variable that is provided at runtime via native binding, the value will be declared by the native binding, not in a script file.
5. Variables declared with the export keyword must only be declared in the global scope.

2.5.3.Struct Declarations🔗

struct-declaration:

declaration-modifiers struct identifier { property-declaration_{opt rep} }

property-declaration

type-expression identifier

type-expression identifier = expression

Description

Declares a struct data type and its properties.

Constraints

1. A struct must not be declared with the either native or const modifier.
2. Struct's must only be declared in the main scope of a source file.
3. Each struct property must have a unique name.
4. Each struct property, if a default value is set, must have an expression whose type is assignable to the property's type.

2.6.Statements🔗

statement:

labeled-statement

declaration

statement-body

statement-body:

block-statement

for-loop-statement

do-while-statement

while-statement

if-statement

return-statement

control-flow-statement

assert-statement

expression

2.6.1.Block Statements🔗

block-statement:

{ statement_{opt rep} }

Description

Declares a list of statements delimited by { and } characters.

Variables declared inside of block statements cannot be referenced from outside of that block.

2.6.2.Loop Flow Control Statements🔗

control-flow-statement:

continue identifier_opt

break identifier_opt

Description

Influences the way a loop executes. Continue statements instruct the loop to move on to the next iteration of execution, while break statements cause the loop to be exited early before its condition can evaluate to a false value.

Control Flow statements can be declared with a label. This label is optional, if no label is specified, the statement changes how the immediate loop behaves. If a label is set, then it controls how the referenced loop behaves.

Constraints

1. Can only be used inside a loop
2. If a label is specified, the label can only reference a label of a loop the statement is inside.

2.6.3.If Statements🔗

if-statement:

if loop-condition-expression statement-body

if loop-condition-expression statement-body else statement-body

Description

Evaluates a condition and executes either a body, an else statement or moves on without executing.

Constraints

1. The if statement's condition must result in a value that can be assigned to a true/false value.

2.6.4.Labeled Statements🔗

labelled-statement:

identifier : for-loop-statement

identifier : do-while-statement

identifier : while-statement

2.6.5.For Loop Statements🔗

for-loop-statement:

for ( variable-declaration ; expression ; expression ) statement-body

Description

Declares a variable and iterates until the 2nd expression, the loop condition evaluates to a false value. After each iteration the third expression is executed as well.

Constraints

1. The for loop's second expression must result in a value that can be evaluated as a boolean.

2.6.6.Do While Loop Statements🔗

do-while-statement:

do block-statement while loop-condition-expression

Description

Evaluates a block of code until the defined loop condition results in a false value. The loop's block will be evaluated at least once until the condition is reached.

Constraints

1. The loop's condition must result in a value that can be evaluated as a boolean.

2.6.7.Regular While Loop Statements🔗

while-statement:

while loop-condition-expression block-statement

Description

Evaluates a block of code until the defined loop condition results in a false value. If the condition results in a false value when the loop is first reached, the loop's block is not evaluated at all.

Constraints

1. The loop's condition must result in a value that can be evaluated as a boolean.

2.6.8.Return Statements🔗

return-statement:

return expression_opt

Description

Causes the current function to halt execution and return to the caller.

Constraints

1. If the return statement is used in a non-void function, it must always return a value.
2. If the return statement is used in a void function, it must never return a value.

2.6.9.Assert Statements🔗

assert-statement:

assert expression

assert expression : expression

Description

The assert statement evaluates an expression and if expression results in a false value, it throws an error with a potentially user-specified error message.

Constraints

1. The expression the assert statement evaluates must always result in a Boolean value.
2. The message expression, if present, must always result in a String value.

Examples

assert true // Does nothing
assert false // Assertion failed!
assert false : "message" // Assertion failed: message

2.7.Script File Statement🔗

script-file-statement:

declaration_rep

3.Interpreter🔗

quickscript's internal compiler will compile source files into an internal bytecode that is then executed by the interpreter, that bytecode will be referred to as the IR (Intermediate Representation).

3.1.Registers🔗

Quickscript is a register based language, and as such will use 64 64bit registers to store values that are being worked with immediately. The following registers are reserved for special purposes:

Return Value Register (register 0)

The Return Value Register, or just "rvr" for short, is the register used to store the return value of function invocations.

Instruction Counter Register (register 1)

The Instruction Counter Register, or just "icr" for short, is the register used to store the index of the current instruction.

3.2.Management🔗

The interpreter will maintain the following internal elements:

Global Type Table

A type table used for referencing types across all scripts loaded into the interpreter.

Global Const String Pool

A string pool that contains every constant string of every loaded script.

Global Instruction Buffer

A buffer of IR instructions from every loaded script.

3.3.Bytecode File Loading🔗

When loading IR bytecode files, the file's instructions are added to the interpreter's instruction buffer. Any jump instructions in the loaded file must then be corrected and offset according to the position in the buffer the instructions were inserted.

All const strings in the loaded file must be placed into the global string pool. If a string already exists in the global string pool, it must not be placed in again. The script's instructions must be rewritten to ensure it's using the already existing string's offset. The same must go for the type table and function tables, which may refernce values in the file's string pool.

Types in the type table must then be loaded into the interpreter's global type table. If a type index in the stored file does not match the type index after a type was placed into the global pool, any references to the old type index must be rewritten to point to the new one.

Finally, all functions table entries, when loaded, must be placed into the interpreter's global function table, if any function's index changes, update the instructions to match.

3.4.Object Memory Layout🔗

This section describes the layout of quickscript objects (structs, arrays and strings) in memory.

3.4.1.Strings and Arrays🔗

There exist two types of arrays and strings: const and non-const. Const versions have no prefixing reference counter that is used to track when the object should be freed. This section will address both strings and arrays, since their layout in memory is identical.

The first bit of an array is the const/non-const determining bit. If it's 0, the value is const and has no reference counter. Otherwise, if the bit is set, the object is reference counted.

3.4.1.1.Const Array Memory Layout🔗

Length prefix (32bit unsigned integer)

A 32bit unsigned integer that tells you the number of elements in the array, not the size in bytes.

Array Data (N bytes of data)

An unspecified number of bytes of data representing the objects stored in the array. Note that since quick script arrays exist for each declared type, the size of this block of data is equal to the length prefix multiplied by the size of an individual element, which is not known if you don't already know it :3

3.4.1.2.Non-Const Array Memory Layout🔗

Reference Counter (32bit unsigned integer)

A 32bit unsigned integer that acts as the reference counter for the object, note that since the first bit is the const/non-const determinant, it is ignored when reading the reference counter's value.

If the reference counter ever reaches 0, the array should be freed as there are no more references to it.

Length prefix (32bit unsigned integer)

Length of the array, same as in const arrays

Array Data (N bytes of data)

Array data, same as in const arrays

3.4.2.Structs🔗

Structs always have a prefixed reference counter, there are no const structs. As such, their memory layout is the following:

Reference counter (32bit unsigned integer)

Reference counter, follows the same rules as non-const array reference counters.

Struct data

The specific layout of structs is static and changes with each struct type. For example, assume a struct of 3 float32 values x, y, and z. This struct's data would have a size of 12 bytes with each property being offset 4 bytes more than the last one from the start of the struct data.

3.5.Function Invocation🔗

Function invocation requires 2 things, a function pointer, and knowledge of what kind of function is being invoked. There should exist at least 2 types of functions:

Local Functions

Local functions are functions declared in the same file as the caller, and as such all they do is index into the file's local function table to find the function. The benefit of this, is that the VM can replace lookup calls with the literal value, or index, or whatever of the function its looking for.

Native Functions

Native functions require a lookup through the VM's function lookup table, which is then given a name and a function signature to look for.

In both cases, a function pointer is returned and the invocation can begin.

Invocation itself begins with a stack frame allocation. The first n bytes being for the returned value (n standing for the stack memory size of the returned type.) The following allocated bytes are for each argument, which are then written to, to initialise the values.

What happens next is dependent on the type of function being invoked.

3.5.1.Local functions🔗

The instruction counter is set to the local function's first instruction and a new stack frame allocated for all the memory of the target function. Note that this stack frame overlaps with the previous frame at the point where the function arguments begin.

The function is then interpreted and its return value written to register 0, aka the rvr register.

3.5.2.Native functions🔗

Although the way the return value and arguments are handled are similar, what happens immediately after the invocation instruction is interpreted is that the native function is called, and after its execution finishes, the returned value is written to the rvr register.

3.6.IR OP Codes🔗

A lot of the following opcodes will refer to some arguments as "registers." This means the argument is an unsigned 8bit integer index for one of the interpreter's registers.

Any operation listed here which is based on a certain memory size is also used for pointer types and floating point types, as for those the numeric type doesn't really matter, they're still just numeric values stored in bits.

3.6.1.General purpose OP Codes🔗

NOP

A no-operation operation that does nothing. This isn't intended to be used or even appear in the outputted IR, but exists as a placeholder for an opcode with a 0 value.

PUSHLINE

Pushes a line number to the interpreters tracker telling it what line of the source it is currently executing, used for when stuff goes wrong or for inspecting the IR and easily comparing it with the source code.

RET

Each function's data is suffxed with this instruction regardless of if a return statement was actually added in the source. It tells the interpreter when to return to the caller up the call the stack.

If there is no call frame to return to after a RET instruction, the program has reached the end of execution and should exit.

JMP, JMPI0 and JMPN0

Are all jump instructions. The first one simply jumps to a different instruction. The other two are conditional jumps. JMPI0 means jump if zero and JMPN0 means jump if not zero.

All instructions take a 32bit unsigned integer as their first argument, which is the next instruction to jump to. The 2nd and 3rd instructions take in a 2nd argument as well: the register value to evaluate is inside.

LOADCONST (has byte-dependent codes)

These are all instructions which take as arguments a register to store a constant value inside and a 2nd argument which is the constant value itself.

MOV

Copy the value in the first register argument into the second register argument.

ASSERT

Takes in a register containing a boolean value, tests the value and if it is false, throws an error with an optionally user-specified error message.

Arguments:

• (uint8) Register containing the assertion expression's result
• (uint8) Register containing the message string

3.6.2.Stack Memory OP Codes🔗

Stack read and write operations come in two different flavors: Relative and Absolute. Absolute reads and writes to and from the global scope of the current script file, while relative instructions read from the start of the last stack frame

SREAD (has byte-dependent codes)

Stack read of a corresponding bit-size from the stack memory. Takes in two arguments:

• The register to store the read value.
• The memory offset to read from. Offsets relative to stack address at the start of the function execution.

SWRITE (has byte-dependent codes)

Stack write of a corresponding bit-size to the stack memory. Takes in two arguments:

• The register to read the value from.
• The memory offset to write to.

STORECONST (has byte-dependent codes)

Write a constant value to stack memory

Arguments:

• (uint32) Offset to write to
• (size-depdendent) Value to write

3.6.3.Global Memory OP Codes🔗

GREAD (has byte-dependent codes)

Global read of a corresponding bit-size from memory. Takes in two arguments:

• The register to store the read value.
• The memory offset to read from. Offsets relative to stack address in the global scope

GWRITE (has byte-dependent codes)

Global variable write of a corresponding bit-size to memory. Takes in two arguments:

• The register to read the value from.
• The memory offset to write to.

GSTORECONST (has byte-dependent codes)

Write a constant value to global memory

Arguments:

• (uint32) Offset to write to
• (size-depdendent) Value to write

3.6.4.Closure OP Codes🔗

Closures are basically just pointers to the stack frames.

GETSTACKPTR

Meaning "Get Stack Pointer," gets the current stack pointer.

Arguments:

• (uint8) Register to write the closure address to.

CREAD (has byte-dependent codes)

Closure read of a corresponding bit-size from memory.

Arguments:

• (uint8) The register containing the closure.
• (uint64) The offset to read the value from in the closure
• (uint8) The register to store the read value.

CWRITE (has byte-dependent codes)

Closure write of a corresponding bit-size to memory.

Arguments:

• (uint8) The register containing the closure.
• (uint64) The offset to write to in the closure
• (uint8) The register containing the value.

3.6.5.Heap Memory OP Codes🔗

READOBJ (has byte-dependent codes)

Instruction for reading a struct property's value. This can only be used for structs as it expects the memory layout of the object its reading from to match a struct's layout.

Arguments:

• (uint8) Register containing the object pointer to read from
• (uint8) Register to store the output of the read value.
• (uint32) The byte offset of the object's data to read from

WRITEOBJ (has byte-dependent codes)

Instruction for writing to a struct's data.

Arguments:

• (uint8) Register containing the object pointer to write to.
• (uint8) Register to read the value from.
• (uint32) The byte offset of the object's data to write to

READIDX (has byte-dependent codes)

Instructions for reading from an array or string (struct or array or string)

Arguments:

• Register containing the "object" to read from (a pointer)
• Register to store the output of the read value.
• Register containing the index to read from

WRITEIDX (has byte-dependent codes)

Instruction for writing to an array.

Arguments:

• Register containing the "object" to write to.
• Register to read the value from.
• Register containing the index to write to

(Note: Strings should never be written to I mean granted the interpreter isn't specified to check if you are, but you know, don't.)

ARRLEN

Read an array object's length.

Arguments:

• (uint8) Register containing the object's pointer
• (uint8) Register to store the length in (uint32)

OBJALLOC

Allocate an object on the heap and store the resulting pointer in a registry.

Arguments:

• (uint8) Register to store the result in.
• (uint32) Type index of the object, must be a struct's type.

ARRAYALLOC

Allocate an array of an object type and store the resulting pointer in a registry.

Arguments:

• (uint8) Register to store the result in.
• (uint32) Size of the array to allocate.
• (uint32) Type index of the array type. Note that this must be the array's type, not the component type.

3.6.6.Function Call Instructions🔗

LFUNCLOOKUP

Stands for "Local Function Lookup."

Arguments:

• (uint32) Local function table index.
• (register) Output register to store the result in.

NFUNCLOOKUP

Stands for "Native Function Lookup."

Arguments:

• (uint32) The type index of the function signature to look for.
• (uint64) The offset of the function's name in the const string pool.
• (register) Output register to store the result in.

SETARGTYPE

Set a value in the interpreter's internal array that contains the types of function arguments.

Arguments:

• (uint32) Argument index
• (uint32) Type index

INVOKE

Function invocation opcodes. See section 3.5. Function Invocation for how functions are invoked.

All function invocation results are stored in register 0, aka the rvr (Return Value Register) register.

Arguments:

• (uint8) The register containing the function pointer

3.6.7.Foreign Symbol OP Codes🔗

These opcodes allow for scripts to read from imported symbols or from binding values or constants.

FREAD

Read a foreign symbol or constant

Arguments:

• (uint8) Register the read value will be placed in
• (uint64) Offset of the symbol name in the const string pool
• (uint32) Type index of the symbol

FWRITE

Write to a foreign symbol

Arguments:

• (uint8) Register the value will be read from
• (uint64) Offset of the symbol name in the const string pool
• (uint32) Type index of the symbol

3.6.8.Conversion Instructions🔗

These are instructions that convert from one number type to another. I will not be listing them here, but they are shown below in section 8. OP Code Table.

3.6.9.Unary instructions🔗

These instructions take in 2 arguments, an input and an output register. Each instruction then performs some operation and places the result in the output register.

BNEGATE

Bitwise negate operation

LNEGATE

Logical negate operation

NEG (has number type dependent operations)

Numeric sign flip operation.

Note that for operations like NEGU8 (Unsigned 8 bit negative op code) the result becomes a signed 8 bit integer.

INC (has number type dependent operations)

Increment a number by 1

DEC (has number type dependent operations)

Decrement a number by 1

3.6.10.Binary Operations🔗

Binary instructions take in 3 arguments, a left-hand-side register argument, a right-hand-side register argument and an output register argument. The instruction then performs an operation on the arguments and places the result in the return register.

I'm not listing all the opcodes in this section as there's many variations of each one. But I will list the types of opcodes.

In the following sections, some instructions are marked as bitwise operations and some as bitwise. The difference is in how the bits are handled. Logical operations are intended for boolean operations, so they function slightly differently. For example, the bitwise OR operation combines the bits in two integers, where as the boolean OR operation outputs a 1 or 0 depending on the value of both operands.

3.6.10.1.Integer-only Binary Operations🔗

These are instructions that are only supported for the integer types, signed and unsigned, and mostly includes bitwise operations

• LSHIFT: Left Shift
• RSHIFT: Right Shift
• BAND: Bitwise AND operation (&)
• BOR: Bitwise OR operation (|)
• BXOR: Bitwise XOR operation (^)

3.6.10.2.Boolean-only Binary Operations🔗

Some overlap in this category with the previous, but these operations function slightly differently.

• LAND: Logical AND operation (&&)
• LOR: Logical OR operation (||)
• LXOR: Logical XOR operation (^)

Note: The use of LXOR or BXOR has to be decided by the compiler depending on the type of the operands involved. This is because they use the same operator (^) in the source code.

3.6.10.3.General Number Binary Operations🔗

This category encompasses mostly mathematical operations for each number type supported by quickscript.

ADD

These operations add two numbers.

SUB

These operations subtract the right argument from the left.

MUL

These operations multiply the left argument with the right.

DIV

These operations divide the left argument by the right.

MOD

Modulo instruction.

POW

These operations raise the left argument to the power of the right

3.6.10.4.Comparison Operations🔗

These are operations that in some way compare the left and right operands

EQ

Equals operator, compares the byte values

EQARR

Equals operator for arrays

EQSTRUCT

Equals operator for structs

NEQ

Not-Equals operator, compares the byte values

NEQARR

Not-Equals operator for arrays

NEQSTRUCT

Not-Equals operator for structs

GT (has number-type-depdendent codes)

Greater-Than operator

GTARR

Greater-Than operator, for arrays

GTE (has number-type-depdendent codes)

Greater-Than-or-Equal-to operator

GTEARR

Greater-Than-or-Equal-to operator, for arrays

LT (has number-type-depdendent codes)

Less-Than operator

LTARR

Less-Than operator, for arrays

LTE (has number-type-depdendent codes)

Less-Than-or-Equal-to operator

LTEARR

Less-Than-or-Equal-to operator, for arrays

3.6.10.5.String/Array Operations🔗

Special binary operation cases carved out for string and array operations.

STRCONCAT

Concatenate a string with a value. This instruction takes in the type index of the right hand side argument as well. This is required to ensure it is correctly converted to a string and then concatenated with the left hand side string.

Arguments:

• (uint8) left-hand-side register, must contain a string pointer
• (uint8) right-hand-side register
• (uint32) Type index of the right-hand-side.
• (uint8) Output register.

STRREP

String repetition OP Code. The right-hand-side register given to this instruction must contain an unsigned integer or a positive integer.

4.Standard Library🔗

This section will just describe all the functions and constants I can think of that should be part of the standard library.

4.1.Constants🔗

PI

3.1415, you know, pi

E

Euler's number

LN10

Natural logarithm of 10

LN2

Natural logarithm of 2

LOG10E

Base-10 logarithm of E

LOG2E

Base-2 logarithm of E

SQRT1_2

Square root of 1 / 2

SQRT2

square root of 2

4.2.Functions🔗

Some functions are defined multiple types for several number types, in those cases the following placeholders will be used:

float

float32 and float64

num

All numeric types

int

Any integer types

I will also omit descriptions from some functions because of how common they are that you can look up the same function in basically any other language and find its definition.

4.2.1.General functions🔗

void printf(string format, args...);

Prints to the standard output, effectively identical to C's printf, except for the ability of quickscript to automatically discern what the argument types are.

println(string format, args...)

Same as printf, but appends a new line character to the end of the print string.

string sformat(string format, args...);

Formats a string in the same way as printf.

4.2.2.Time and date functions🔗

currentTimeMillis()

Returns the current UNIX timestamp.

4.2.3.Math functions🔗

float sqrt(float x);

Square root function

float cbrt(float x);

Cube root function

num abs(num x);

float acos(float x);

float acosh(float x);

float asin(float x);

float asinh(float x);

float atan(float x);

float atan2(float x, float y);

float atanh(float x);

float ceil(float x);

int clz(int x);

float cos(float x);

float cosh(float x);

float hypot(float... values);

float log(float x);

float log10(float x);

float log1p(float x);

float log2(float x);

num max(num... values);

num min(num... values);

float round(float x);

float round(float x, uint32 precision);

int8 sign(num x);

float sin(float x);

float sinh(float x);

float tan(float x);

4.3.Structs🔗

5.Type Checker And Semantic Analysis🔗

After a source file is parsed into a syntax tree (Abstract Syntax Tree, which I'll be calling the AST) the types referenced and declared in the code must be resolved for the compilation step to take place. This is so the compiler knows how to handle variables and how much space they take up.

Type resolution is the process of resolving referenced type names into their actual types and resolving what type an expression returns. The validation process involving checking that those types are used correctly, for example making sure that a variable declaration's value type can be assigned to the declared variable's type.

The beginning of this process is the most difficult, as it requires somehow simultaneously knowing both the functions available in a file, and all the declared types (structs.) Which obviously isn't possible all the time. As a general rule, types tend to be declared at the tops of files and functions take up the rest of the space, but this shouldn't be enforced. Stylistic decisions shouldn't determine behavior.

To further complicate matters, structs can have properties which reference other structs. Types have to somehow know every other type that exists at all times without actually knowing it. What...

5.1.Type Resolution🔗

Type resolution will run in a couple passes, specified here:

Initial Struct Pass

Iterate over all declared structs and create a ScriptStructType with the names of each property added to the type, but the types of each property left null.

Second Struct Pass

Iterate over all declared structs and now initialize the types of their properties. These two struct passes should ensure that the declaration order of the structs is irrelevant.

Function Signature pass

Iterate over every declared top level function and create a FunctionSignature object to represent its signature and then place the function into the global scope's symbol table so it can be referenced later.

With these initial steps completed, the script file should be able to resolve every type and function without issue, provided all the symbols referenced in the script's code are valid.

Code Pass

This is the pass which finally involves iterating over the individual expressions and statements that actually make up the code that is executed and resolving types and checking that the code is valid. More on this in the next section.

5.2.Semantic Analysis🔗

This step just involves going through the code and going "yeah na chief, u did it wrong." It's a programmatic nitpicker.

Basically go through the statements and expressions and make sure the operations are valid, the functions being called actually exist, the properties on structs, arrays and strings being accessed exist and check for attempted writes to const values.

As this step is being executed, the type table needs to be filled out with declared types, function signatures of nested functions and symbols created for local variables and functions.

6.Semantic Transformation🔗

This is the step right before IR compilation. Generally, this prepares the parsed code for compilation.

6.1.Inlining🔗

Mostly this is just literal value inlining. Basically turn 2 + 2 into 4, literally. It also involves taking constant variables expressions and inlining their values since using a value is easier than loading one from the stack or from heap memory.

Additionally, any "constexpr" functions (Functions which can be evaluated during compilation time) are inlined and their calls replaced with the result of their values.

Note that this step is intended to be recursive, meaning that, for example, after a function is inlined, any expressions it was used in should be optimized after the inlining. Potentially turning the following code:

uint32 f() {
  return 3
}
void main() {
  const uint32 x = 2 * f()
}

void main() {
  const uint32 x = 6
}

In the future, there might be a possibility of exporting functions or types or global variables. If the language does get that far, then under no circumstance can exported functions or constants be omitted from the compiled IR. Non-exported, however, can be dropped if either no calls exist or all calls were inlined.

The same inlining applies to constant variables with an inline-able value. Note that in the future, if imported constants become a thing, they cannot be inlined as their type may be known at compile time, but their value should not be assumed to be the same during run time. This may be because of differences in versions between compilation and run time, or whatever.

The following sections describe the specifics of inlining.

6.1.1.Binary operation evaluation🔗

Simple, turn math operations into their values: 2 * 2 becomes 4.

Note that this needs to follow the rules of binary expression combining... which I haven't specified here. Of course. Well I can't be bothered either, so here's a link to the source code comment I wrote (Valid at time of writing): src/analysis/analyzer.cc#getOpResultType

6.1.2.String operation evaluation🔗

Two operators can be used with strings: The concatenation operator (+) and the repetition operator (*)

If there are any instances of a string being concatenated with any primitive literal value or another string, then the expression should be inlined, for example: "value: " + 12 becomes "value: 12"

The same applies to the repetition operator. "-" * 3 becomes "---", for example.

6.1.3.Unary operation evaluation🔗

Turn unary operations into their literal values. In the case of the unary positive operator +<expression> Just get rid of the unary operation and return the target expression lmao, useless operator.

Depending on the context of the unary operation, the expression can be changed even if the value is not a literal. Mostly this means flipping postfix operations to be prefix operations.

For example:

for (uint32 i = 0; i < 10; i++) {/* code */}

for (uint32 i = 0; i < 10; ++i) {/* code */}

This is because prefix operators save ONE WHOLE instruction, oh my gods.

6.1.4.Drop "pointless" statements🔗

1
2
3
4
5
6
7

void main() {
  int a = 0
  a + 1 // Drop this
  -a // Drop this
  ++a // Do NOT drop this
  return a
}

The definition of pointless here goes beyond just expression statements, "pointless" here also refers to variables and functions are unused, they should be dropped, again, unless it's an exported symbol, then it can't be dropped.

Loops with empty bodies, or bodies made up entirely of "pointless" statements should be dropped as well.

6.1.5.Drop zero values🔗

This means things like

uint32 a = 0

uint32 a

This is because the stack allocator will, after a stack frame has been allocated, zero all allocated bytes, so writing a 0 to the stack after allocation is pointless.

This also applies to struct member default values, 0 values should be dropped there too, as the heap allocator should zero all bytes when a heap allocation is performed.

6.1.6.Inline conditional statements with literal conditions🔗

This inlining also applies to while loops, but only if the constant is known to be false.

6.1.7.Advanced inlining🔗

This is in its own section mostly because I can see the logic behind the inlining, but I don't know how to implement it personally yet.

6.1.7.1.Function inlining🔗

This is simpler for some functions than others, for example, all of these functions can be considered inline-able:

uint32 f() { return 0 }

uint32 f(uint32 a) { return a }

uint32 f(uint32 a) { return a * 2 }

uint32 f(uint32 a) {
  for (uint32 i = 0; i < 10; i++) { a += i }
  return a
}

uint32 f() {
  uint32 a
  for (uint32 i = 0; i < 10; i++) { a += i }
  return a
}

uint32 f() {
  uint32 y() {
    return 0
  }
  uint32 a = y()
  for (uint32 i = 0; i < 10; i++) { a += i }
  return a
}

Mostly this inlining happens recursively, as stated before, meaning if every statement inside a function is inlined away or dropped, it should make determining if a function can be inlined easier.

6.1.7.2.Constant inlining🔗

As I might've mentioned before, only locally declared constants can be inlined. Locally in this case meaning either declared within a function's body or locally inside the file.

This step should be easier than function inlining: If a constant has a literal value, replace references to that constant with its literal value.

6.2.Function flattening🔗

Take all nested functions and flatten them so they are all top level functions.

Function names should be flattened by appending their nesting function(s) name(s) to the function name, separated by a # symbol, since that symbol cannot appear in regular identifiers. Additionally, the name should be prefixed with a single % character to prevent potential future conflicts with flattened struct methods.

Function flattening will apply to struct methods, if the language ever gets that far.

6.3.Constructor creation🔗

Each struct will have an implicitly created constructor whose job is to allocate the memory for the struct and initialize its members' default values.

For constructor creation, the first statement will be a lexical declaration statement with a special expression as a value: MallocExpr. This expression can only be inserted by the semantic transformer. The rest of the "function" is just the non-zero default property initializers.

6.4.Global Scope Init Function Creation🔗

When a script file's execution begins, a stack frame is allocated for its global scope variables, but as mentioned before, stack frames are zeroed on allocation, meaning the actual values of global scope variables have to be initialised.

This is where the generated <finit> function comes in. It initializes all global scope variables to their values and then calls the main function written by the user, if there is one.

The generated function will take in a string array (the program's command line arguments) and return a 32bit integer, the program return code.

Depending on the signature of the declared main function, the program arguments may or may not be passed to the main function. Similarly, if the main function returns void, instead of a 32bit integer, then code 0 is always returned. Otherwise, the main function's return value is returned.

The <finit> function will exist for all compiled quickscript files that have global variables, even those without a main function.

6.5.Process Scopes🔗

This involves going through every function and flattening the scopes so all loops if statements and blocks use the same stack frame, just offset differently.

The analyzer creates a different scope and stack for each loop and if statement and block so its variables are isolated from the function scope around it.

When flattening scopes, each function should only use one scope and stack. No need to worry about variables overlapping because after a loop finishes, its variables are useless so they can be discarded and the memory they occupied can be reused.

This step also involves calculating the offset of each variable in a stack frame.

7.Structure of the Bytecode file🔗

This section details the structure of the bytecode file and bytecode data output by the compiler.

7.1.Header🔗

The following is a list of what the bytes of the bytecode file mean, in order.

File Prefix (ASCII String)

ASCII string with value "quickscript"

File Version (unsigned 16bit integer)

Version counter for tracking which compiler version compiled this file. Older interpreters cannot run files compiled by newer versions of the compiler.

Currently, before the first version of the interpreter has even been finished, this will be 0.

String Const Pool Offset and Size (2 unsigned 64bit integers)

Offset of the string const pool from the start of the file, and its size in bytes.

Type Table Offset and Size (2 unsigned 64bit integers)

Offset of the file's type table from the start of the file and its size in bytes.

Function Table Offset and Size (2 unsigned 64bit integers)

Offset of the file's function table from the start of the file and its size in bytes.

Instruction Buffer Offset and Size (2 unsigned 64bit integers)

Offset of the instructions from the start of the file and its size in bytes.

Global Scope Size (unsigned 64bit integer)

Size of the script file's global scope, in bytes.

Entrypoint function index (unsigned 64bit integer)

Index of the entrypoint function in the file's function table

7.2.Const String Pool🔗

An array of QS Strings (32bit length prefix followed by UTF-8 encoded character data) that make up identifiers, string literals, function names and type names.

7.3.Type Table🔗

An array of Type declarations that were declared in the source file. Each entry is preceded by a 32bit unsigned integer, the index of the type, and by an 8bit unsigned integer, the type of the entry. The following types are supported:

ARRAY (0x0)

Array type, the following 32bits after the type byte is the index of the component type in the table.

STRUCT (0x1)

Structure type, the layout of this type's data are specified further down.

NSTRUCT (0x2)

Native Struct, indicates a struct that has been declared by the compiler itself and as such is only referenced by the script, not declared.

FUNCSIGN (0x3)

Function signature type, data layout specified further down

7.3.1.Reserved type indexes🔗

7.3.2.Struct Types🔗

Name offset (uint64)

The offset of the name of the struct in the const string pool.

Constructor location (uint32)

Index of the constructor function of the struct in the function table.

Property Count (uint32)

The number of properties this struct has

Property Array (propertydecl[])

An array of the following values:

Name offset (uint64)

Same as the struct's name offset, address of the name inside the const string pool

Offset (uint32)

Offset of the property's data in memory from the start of the struct's memory

Type Index (uint32)

The type index of the property's type

7.3.3.Function Signatures🔗

Return Type (typeindex)

Return type's type index

Argument Count (uint32)

Number of arguments the signature has

Varargs (bool)

A 1 or 0 byte to indicate if the last argument is a variadic argument.

Arguments (typeindex[])

Array of type indexes, each one indicating the corresponding argument's type.

7.4.Function Table🔗

An array of functions, their names and their starting offsets. Each entry in this array uses the following values:

Name Offset

Offset of the function name in the string const pool.

Note that for constructors, the function name will be something like StructName.<constructor>. And for file init functions it will be <finit>.

Start

The offset of where the function starts in memory

Signature (typeindex)

Index of the function signature type in the type table.

7.5.Instruction Array🔗

Uninterrupted array of compressed IR instructions. Compressed here meaning that the IR instructions are saved by stripping away the empty padding used to ensure all instructions have the same size.

8.OP Code Table🔗