mirror of
https://git.planet-casio.com/Lephenixnoir/JustUI.git
synced 2024-12-28 04:23:40 +01:00
juic: WIP
This commit is contained in:
parent
550c08e200
commit
24f3e14061
9 changed files with 1647 additions and 0 deletions
2
.gitignore
vendored
2
.gitignore
vendored
|
@ -7,3 +7,5 @@ giteapc-config-*.make
|
|||
|
||||
# Developer's files
|
||||
*.sublime-*
|
||||
|
||||
__pycache__
|
||||
|
|
238
doc/jui-lang.txt
Normal file
238
doc/jui-lang.txt
Normal file
|
@ -0,0 +1,238 @@
|
|||
## Introduction
|
||||
|
||||
JustUI includes a UI specification language which can be used to specify user interfaces in a declarative/functional manner, and generates C code for building the user interface and basic interactions. It doesn't remove the need to write custom C code for most of the dynamic aspects of the interface, but it automates its construction and sometimes a bit more.
|
||||
|
||||
The language is a purely functional, dynamically-typed language with call-by-need evaluation. Its main feature is _record types_, a structure data type which comes with convenient construction tools and supports flexible kinds of (non-recursive) self-references.
|
||||
|
||||
---
|
||||
|
||||
## Types of data
|
||||
|
||||
- The null constant (`null`) and booleans (`true`, `false`);
|
||||
- Primitive types: arbitrary-precision integers (`42`, `-666`), floats (`-73.0f`, `1729.0`), strings (`"JustUI?\n"`);
|
||||
- References to C/C++ objects, treated as pure text (`@img_title`, `@mywidget::draw`);
|
||||
- Collections: lists.
|
||||
- Functions: closures (carry their own context), but not curried.
|
||||
- Records.
|
||||
|
||||
Another kind of “data” that isn't actually a language value is record types themselves, e.g. the `jlabel` type which is used to construct records that describe a label widget.
|
||||
|
||||
> Note: they couldn't easily be first-class because they're associated with a given interpretation of inline parameters, which makes no visual sense unless the name is explicit. Computing a record type then instantiating it with inline parameters is way too confusing. If it's just used in an update, compute the instance.
|
||||
|
||||
---
|
||||
|
||||
## Semantics: Basics
|
||||
|
||||
Expressions for primitive data types are pure.
|
||||
|
||||
The following builtin operators are defined (in decreasing order of precedence).
|
||||
|
||||
- `!`, `-`, `...` (unary)
|
||||
- `*`, `/`, `%` (left associative)
|
||||
- `+`, `-` (left associative)
|
||||
- `==`, `!=`, `<`, `>`, `<=`, `>=` (not associative)
|
||||
- `&&` (left associative)
|
||||
- `||` (left associative)
|
||||
- `f <| x` (left associative)
|
||||
- `x | f` (right associative)
|
||||
|
||||
Arithmetic/logical operators have their usual meaning as in Python/C. The other operators, `...x`, `f <| x` and `x | f`, are described later.
|
||||
|
||||
Interesting expressions start at `let`-bindings and functions. Within a scope (described later), values can be defined with a `let`-binding:
|
||||
|
||||
```
|
||||
let x = 3 * 14 + 31;
|
||||
```
|
||||
|
||||
And functions can be defined with `fun`:
|
||||
|
||||
```
|
||||
fun add(x, y) = x + y;
|
||||
```
|
||||
|
||||
The operators `f <| x` and `x | f` are left-sided and right-sided function application, respectively. They are both equivalent to `f(x)` and are mainly useful because of their syntax: they don't need parentheses and are associative, which makes chaining much easier to read.
|
||||
|
||||
Functions can take variadic parameters with `...`. There can only be zero or one variadic parameter in a function; it there is one, it must be at the end of the parameter list. The variadic parameter will collect any number of extra arguments (including zero) in a list value.
|
||||
|
||||
```
|
||||
fun multiply_sum(factor, ...floats) = factor * sum(floats);
|
||||
```
|
||||
|
||||
Conversely, arguments to a call can be produced from a list by using the pack expansion operator `...`.
|
||||
|
||||
```
|
||||
let args = [3.2, 4.0, 5.0];
|
||||
multiply_sum(...args, 8.0);
|
||||
/* equivalent to multiply_sum(3.2, 4.0, 5.0, 8.0) = 54.4 */
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Semantics: Record types
|
||||
|
||||
Record types are dictionary-like structures with key/value pairs. The primary motivation for records is to represent widgets.
|
||||
|
||||
Records are built using _record constructors_, which are simply the names of record types. Using a record type's name as an expression produces the _default record_ of that type, which is usually empty. Most of the time however, the record is further specified by adding fields in braces:
|
||||
|
||||
```
|
||||
/* Default record of type jwidget */
|
||||
jwidget;
|
||||
/* Default jwidget with the height set to 20 */
|
||||
jwidget { height: 20; };
|
||||
```
|
||||
|
||||
`jwidget` also has a children field, which can be used to add new widgets in it.
|
||||
|
||||
```
|
||||
jwidget {
|
||||
children: [
|
||||
jlabel { text: "Hello" },
|
||||
jlabel { text: "World" }
|
||||
];
|
||||
};
|
||||
```
|
||||
|
||||
However, this syntax is a bit cumbersome. Record types can take _direct_ parameters, which are raw values not prefixed by a field name. `jwidget` handles its direct parameters by adding the to its `children` field, so the previous example can be shortened to:
|
||||
|
||||
```
|
||||
jwidget {
|
||||
jlabel { text: "Hello" };
|
||||
jlabel { text: "World" };
|
||||
};
|
||||
```
|
||||
|
||||
In fact, `jlabel` also accepts a single direct parameter which is its string, so the example can be further simplified down to:
|
||||
|
||||
```
|
||||
jwidget {
|
||||
jlabel { "Hello" };
|
||||
jlabel { "World" };
|
||||
};
|
||||
```
|
||||
|
||||
A constructed record can further be updated with the `<{ ... }` operator, which can override fields. Direct parameters cannot be specified during an update, because direct parameters are literal arguments to the construction function, so they no longer exist once the construction is finished. A typical example is functions to apply common properties on widgets.
|
||||
|
||||
```
|
||||
/* Make any widget size 80x20 */
|
||||
fun mysize(w) = w <{ width: 80; height: 20 };
|
||||
/* Widget with three labels of that size */
|
||||
jwidget {
|
||||
mysize(jlabel { "Hello" }); // standard function call
|
||||
mysize <| jlabel { "again" }; // left-side function call
|
||||
jlabel { "world!" } | mysize; // right-side function call
|
||||
```
|
||||
|
||||
Such functions are often called on direct record parameters (when they represent children of existing widgets), in which case the function call operators `<|` and `|` are useful for improving readability.
|
||||
|
||||
When updating a record, other fields can be referred to by using the `this` keyword, which represents the object being updated. Fields are updated in-order, and `this` always reflects the latest value of fields.
|
||||
|
||||
```
|
||||
fun mysize(w) = w <{
|
||||
margin: this.width - 20; // refers to original width of w
|
||||
width: this.width - 40; // refers to original width of w
|
||||
height: this.width; // refers to updated width
|
||||
};
|
||||
```
|
||||
|
||||
TODO: Custom record types (`rec`) with direct of parameters: children, string for labels, tabs for gscreen...
|
||||
|
||||
TODO: Evaluation order of `this` in record constructors, cycle resolution through call-by-need (arguments and field values are thunked).
|
||||
|
||||
TODO: `set` statement modifies a constructor in the current scope.
|
||||
|
||||
TODO: Pack expansion for inline parameters
|
||||
|
||||
---
|
||||
|
||||
## Generating user interfaces
|
||||
|
||||
The UI file is interpreted statically. It produces a hierarchy of elements for
|
||||
which scene setup code, "signal connections" and access to inner widgets are
|
||||
provided.
|
||||
|
||||
---
|
||||
|
||||
BIG TODO:
|
||||
- Don't have a scope and ability to let- or fun-declare inside function body!!
|
||||
- `rec x() = y {}; x {}` is it of record type `x` or `y`? How to say that `x`
|
||||
is based on `y`?
|
||||
|
||||
IMPLEMENTATION TODO:
|
||||
- List syntax
|
||||
- List manip' (including for loop)
|
||||
|
||||
TODO:
|
||||
|
||||
The syntax for assigning at labels is a bit dodgy at the moment. It should be
|
||||
`@ident: value` but we already use that for record entry def + implicit child.
|
||||
-> Competing methods for accessing fields: we already have `.name`, labels are
|
||||
thus useful for accessing inline-children
|
||||
-> But labels were supposed to be for runtime, this is mixing things up
|
||||
-> How should labels work really?
|
||||
-> TODO
|
||||
|
||||
- Need ability to reference sub-images in order to assign sprites to BGs
|
||||
-> builtin sub-image record type
|
||||
- Need to have global parameters to control fx/cg
|
||||
-> function-based interface param("FX")
|
||||
- "Control-flow": for(each) loops, what else needed for basic ops?
|
||||
|
||||
Limitation of lazy record construction: if you use a function in the middle of
|
||||
a record construction, the record update `<{ }` will force the evaluation of
|
||||
fields even though it would be possible not to force that.
|
||||
|
||||
```
|
||||
fun f(e) = e <{ width: e.width + 10 };
|
||||
rec r1(x) = f <| base { width: x };
|
||||
rec r2(x) = r1 { height: this.width; x };
|
||||
r2 { 10; }
|
||||
```
|
||||
|
||||
Capturing "this" in a local variable to get a snapshot is required, otherwise
|
||||
we can't refer to parents.
|
||||
|
||||
ability to define nice operators?
|
||||
|
||||
`fun "<|>"(...elems) = hbox <| jwidget { ...elems };`
|
||||
|
||||
(priority/associativity difficult)
|
||||
|
||||
---
|
||||
|
||||
Notes on resolved topics:
|
||||
|
||||
Call-by-need to allow non-recursive dependencies in record constructors.
|
||||
|
||||
Semantics of "this" in evaluation of rec:
|
||||
-> gscreen { x: 1; y: 2; z; u; v; a: 2; c; }
|
||||
-> Fact #1: New fields like x: 1; y: 2 must be defined in terms of base fields
|
||||
of gscreen {}, otherwise it's inconsistent with record update
|
||||
-> Fact #2: Parameters may or may not be needed to specify base fields
|
||||
-> Build a "future context" which tells how to map fields of "this" to either
|
||||
existing attributes or thunks of that expression, and then use that context
|
||||
during the evaluation of the argument thunk.
|
||||
-> OK.
|
||||
|
||||
Static is the right choice, even if gintctl might add stuff dynamically
|
||||
-> In Azur it will almost entirely be static, we want the ability to optimize
|
||||
things by precomputing.
|
||||
-> As long as the interface is well specified (i.e. the added tabs are children
|
||||
of the named child "stack") we can always add some C
|
||||
-> The jui-lang file will mostly initialize, we can minimize interference with
|
||||
runtime dynamics, hopefully
|
||||
|
||||
---
|
||||
|
||||
rogue life bindings UI example?
|
||||
|
||||
let PanelFrame = rectangle {
|
||||
bg: @paintPanelFrame
|
||||
padding: 16px
|
||||
}
|
||||
let BindingsWindow = vbox <| PanelFrame {
|
||||
jlabel { "Select bindings" }
|
||||
|
||||
(Checkbox { bg0: "img1a"; bg1: "img1b" } <|>
|
||||
Checkbox { bg0: "img2a"; bg1: "img2b" } <|>
|
||||
Checkbox { bg0: "img3a"; bg1: "img3b" })
|
||||
}
|
0
juic/__init__.py
Normal file
0
juic/__init__.py
Normal file
20
juic/builtins.py
Normal file
20
juic/builtins.py
Normal file
|
@ -0,0 +1,20 @@
|
|||
from juic.datatypes import *
|
||||
from juic.eval import *
|
||||
|
||||
def juiPrint(*args):
|
||||
print("[print]", *args)
|
||||
|
||||
def juiLen(x):
|
||||
if type(x) not in [str, list]:
|
||||
raise JuiTypeError("len")
|
||||
return len(x)
|
||||
|
||||
builtinClosure = Closure(parent=None, scope=MutableScopeData(defs={
|
||||
"print": (0, BuiltinFunction(juiPrint)),
|
||||
"len": (0, BuiltinFunction(juiLen)),
|
||||
|
||||
# TODO: Remove the built-in record type "record" used for testing
|
||||
"record": (0, RecordType.makePlainRecordType()),
|
||||
"subrecord": (0, RecordType.makePlainRecordType()),
|
||||
|
||||
}, timestamp=1))
|
101
juic/datatypes.py
Normal file
101
juic/datatypes.py
Normal file
|
@ -0,0 +1,101 @@
|
|||
from dataclasses import dataclass, field
|
||||
from typing import Callable, Union
|
||||
|
||||
# Mapping of Jui types to Python types:
|
||||
# Null None
|
||||
# Booleans bool
|
||||
# Integers int
|
||||
# Floats float
|
||||
# Strings str
|
||||
# C/C++ refs CXXQualid
|
||||
# List list
|
||||
# "this" ThisRef
|
||||
# Functions Function, BuiltinFunction
|
||||
# Record types RecordType
|
||||
# Records Record
|
||||
|
||||
@dataclass
|
||||
class CXXQualid:
|
||||
qualid: str
|
||||
|
||||
@dataclass
|
||||
class BuiltinFunction:
|
||||
func: Callable
|
||||
|
||||
@dataclass
|
||||
class Function:
|
||||
# Context in which the function gets evaluated
|
||||
closure: "juic.eval.Closure"
|
||||
# Expression node to evaluate when calling the function
|
||||
body: "juic.parser.Node"
|
||||
# Parameter names, must all be unique. May be empty
|
||||
params: list[str] = field(default_factory=[])
|
||||
# Name of variadic argument if one; must also be unique
|
||||
variadic: str | None = None
|
||||
|
||||
@dataclass
|
||||
class RecordType:
|
||||
# TODO: Record prototypes?
|
||||
|
||||
@staticmethod
|
||||
def makePlainRecordType():
|
||||
return RecordType()
|
||||
|
||||
# @dataclass
|
||||
# class RecordCtor:
|
||||
# # Context in which the record body is evaluated
|
||||
# closure: "juic.eval.Closure"
|
||||
# # List of entries to produce, of type `REC_ATTR` or `REC_VALUE`
|
||||
# entries: list["juic.parser.Node"]
|
||||
# # Parameter names, must all be unique. May be empty
|
||||
# params: list[str] = field(default_factory=[])
|
||||
# # Name of variadic argument if one; must also be unique
|
||||
# variadic: str | None = None
|
||||
|
||||
# @staticmethod
|
||||
# def makePlainRecordCtor():
|
||||
# return RecordCtor(closure=None, entries=[], params=[], variadic=None)
|
||||
|
||||
@dataclass
|
||||
class Record:
|
||||
# A record type if it's pure, or a thunk with the base object otherwise.
|
||||
base: Union[RecordType, "juic.eval.Thunk"]
|
||||
# Standard key-value attribute pairs
|
||||
attr: dict[str, "juic.value.Thunk"]
|
||||
# Children elements
|
||||
children: list["juic.value.Thunk"]
|
||||
# TODO: Keep track of variables that are not fields, i.e. "methods"?
|
||||
# scope: dict[str, "JuiValue"]
|
||||
# TODO: Labels
|
||||
# labels: dict[str, "JuiValue"]
|
||||
|
||||
JuiValue = Union[None, bool, int, float, str, CXXQualid, list,
|
||||
"juic.eval.PartialRecordSnapshot", BuiltinFunction, Function,
|
||||
RecordType, Record]
|
||||
|
||||
def juiIsArith(value):
|
||||
return type(value) in [bool, int, float]
|
||||
|
||||
def juiIsAddable(value):
|
||||
return juiIsArith(value) or type(value) in [str, list]
|
||||
|
||||
def juiIsComparable(value):
|
||||
return type(value) in [None, bool, int, float, str]
|
||||
|
||||
def juiIsLogical(value):
|
||||
return type(value) == bool
|
||||
|
||||
def juiIsUnpackable(value):
|
||||
return type(value) == list
|
||||
|
||||
def juiIsCallable(value):
|
||||
return type(value) in [BuiltinFunction, Function]
|
||||
|
||||
def juiIsProjectable(value):
|
||||
return type(value) in [ThisRef, Record]
|
||||
|
||||
def juiIsConstructible(value):
|
||||
return type(value) in [RecordType, Function]
|
||||
|
||||
def juiIsRecordUpdatable(value):
|
||||
return type(value) == Record
|
611
juic/eval.py
Normal file
611
juic/eval.py
Normal file
|
@ -0,0 +1,611 @@
|
|||
from dataclasses import dataclass, field
|
||||
from typing import Tuple, Union
|
||||
|
||||
import juic.parser
|
||||
from juic.parser import Node
|
||||
from juic.datatypes import *
|
||||
|
||||
# Storage of a scope's internal data, including all definitions in the scope.
|
||||
# The data in this object is never copied, so for instance, the name -> value
|
||||
# mappings are always unique and can only be found in the relevant's scope
|
||||
# unique MutableScopeData instance. This data is, of course, not constant,
|
||||
# which makes it unsuitable for building closures. The Closure type captures an
|
||||
# immutable snapshot of a scope's contents at one point in time.
|
||||
@dataclass
|
||||
class MutableScopeData:
|
||||
# List of definitions. Each value has a timestamp integer that indicates
|
||||
# when it was added; this allows snapshots to ignore definitions newer than
|
||||
# the snapshot by simply comparing timestamp values. The value itself is
|
||||
# either a thunk (evaluated or not), or a runtime value.
|
||||
defs: dict[str, Tuple[int, Union[JuiValue, "Thunk"]]] = \
|
||||
field(default_factory=dict)
|
||||
|
||||
# Current timestamp, changed every time a value is added.
|
||||
timestamp: int = 0
|
||||
|
||||
def addDefinition(self, name: str, content: Union[JuiValue, "Thunk"]):
|
||||
if name in self.defs:
|
||||
raise JuiRuntimeError(f"{name} already defined in current scope")
|
||||
self.defs[name] = (self.timestamp, content)
|
||||
self.timestamp += 1
|
||||
|
||||
def lookup(self, name: str, maxTimestamp: int = -1) \
|
||||
-> Union[JuiValue, "Thunk", None]:
|
||||
if name not in self.defs:
|
||||
return None
|
||||
if maxTimestamp >= 0 and self.defs[name][0] >= maxTimestamp:
|
||||
return None
|
||||
return self.defs[name][1]
|
||||
|
||||
def dump(self):
|
||||
digits = len(str(self.timestamp))
|
||||
print(f"Scope: timestamp={self.timestamp}, defs:")
|
||||
for name, (ts, content) in self.defs.items():
|
||||
print(f" {ts: >{digits}} {name}:", juiValueString(content))
|
||||
|
||||
# An immutable snapshot of all definitions that can be referred to at a given
|
||||
# point in time. This specifies the interpretation of all defined identifiers
|
||||
# but not of the "this" keyword.
|
||||
@dataclass
|
||||
class Closure:
|
||||
# Parent snapshot on which this one is based, which defines the contents of
|
||||
# surrounding scopes at that time.
|
||||
parent: Union["Closure", None]
|
||||
# Reference to the scope in which the closure was created.
|
||||
scope: MutableScopeData
|
||||
# Timestamp at which the closure was created. Definitions from `scope`
|
||||
# created later than the closure will be ignored during name lookup.
|
||||
timestamp: int = field(init=False)
|
||||
|
||||
def __post_init__(self):
|
||||
# Automatically get the timestamp at the time of creation
|
||||
self.timestamp = self.scope.timestamp
|
||||
|
||||
def lookup(self, name: str) -> Union[JuiValue, "Thunk", None]:
|
||||
value = self.scope.lookup(name, self.timestamp)
|
||||
if value is not None or self.parent is None:
|
||||
return value
|
||||
return self.parent.lookup(name)
|
||||
|
||||
def dump(self, level=0):
|
||||
print(f"Closure at depth {level}: timestamp={self.timestamp}:")
|
||||
self.scope.dump()
|
||||
if self.parent is not None:
|
||||
self.parent.dump(level+1)
|
||||
|
||||
# Thunk representing a suspended/lazy computation. Thunks are evaluated when
|
||||
# needed (which is the call-by-need evaluation strategy). They are always
|
||||
# interpreted in the context of the associated closure.
|
||||
#
|
||||
# While the object is technically mutable, the result of the computation
|
||||
# (whether a value or a cyclic invalid result) is entirely determined by the
|
||||
# expression AST and closure, so the result is just a lazily computed/memoized
|
||||
# member. That said, the result might not exist if there is a cyclic dependency
|
||||
# between this and other thunks referenced by the closure.
|
||||
@dataclass
|
||||
class Thunk:
|
||||
# Expression AST that the thunk evaluates to
|
||||
ast: Node
|
||||
# Closure in which the thunk can be evaluated, plus the definition for
|
||||
# "this"; if there is one, it is always a PRS
|
||||
closure: Closure
|
||||
thisReference: Union[None, "PartialRecordSnapshot"] = None
|
||||
|
||||
# Whether the thunk has been evaluated yet
|
||||
evaluated: bool = False
|
||||
# Whether the thunk has been evaluated to an invalid result because of a
|
||||
# cyclic dependency with other thunks captured by the closure
|
||||
invalid: bool = False
|
||||
# If the thunk has been evaluated and is not invalid, resulting value. None
|
||||
# is a possible resulting value (`null` in the language); do not use this
|
||||
# field to determine whether the thunk has been successfully evaluted.
|
||||
result: JuiValue = None
|
||||
|
||||
# Whether the thunk is currently under evaluation
|
||||
_running: bool = False
|
||||
|
||||
# Object representing the interpretation of a partially-built record. In a
|
||||
# record construction or update operation, fields of `this` refer to the latest
|
||||
# definition before the point of use, if there is one, or the definition in the
|
||||
# underlying record otherwise, if there is one. Because the use of record
|
||||
# constructor arguments can create forwards dependencies, all these definitions
|
||||
# are thunked and computed lazily in a somewhat unpredictable order. A partial
|
||||
# record snapshot fully specifies the interpretation of all fields of a
|
||||
# partially-constructed record (i.e. `this`) at the location of one record
|
||||
# entry. It does so by mapping defined field names to the correct thunks. This
|
||||
# object is how dependencies between fields are discovered and followed.
|
||||
@dataclass
|
||||
class PartialRecordSnapshot:
|
||||
# Mapping from fields of `this` to thunks
|
||||
fieldThunks: dict[str, Thunk] = field(default_factory=dict)
|
||||
# Base object for fields not captured in `fieldThunks`
|
||||
base: None | Thunk = None
|
||||
|
||||
def copy(self):
|
||||
return PartialRecordSnapshot(self.fieldThunks.copy(), self.base)
|
||||
|
||||
# TODO: Make this a general value printing function
|
||||
def juiValueString(v):
|
||||
match v:
|
||||
case None:
|
||||
return "null"
|
||||
case bool():
|
||||
return str(v).lower()
|
||||
case int() | float():
|
||||
return str(v)
|
||||
case str():
|
||||
return repr(v)
|
||||
case CXXQualid():
|
||||
return "&" + v.qualid
|
||||
case list():
|
||||
return "[" + ", ".join(juiValueString(x) for x in v) + "]"
|
||||
case BuiltinFunction():
|
||||
return str(v)
|
||||
case Function():
|
||||
p = v.params + (["..." + v.variadic] if v.variadic else [])
|
||||
s = "fun(" + ", ".join(p) + ") => " + str(v.body)
|
||||
s += " (in some closure)"
|
||||
return s
|
||||
case RecordType():
|
||||
return "<RecordType>"
|
||||
case Record() as r:
|
||||
s = juiValueString(r.base) + " " + "{"
|
||||
s += "; ".join(x + ": " + juiValueString(y) for x, y in r.attr.items())
|
||||
s += "; ".join(juiValueString(x) for x in r.children)
|
||||
return s + "}"
|
||||
raise NotImplementedError
|
||||
case Thunk() as th:
|
||||
s = "Thunk("
|
||||
if th._running:
|
||||
s += "running, "
|
||||
if th.evaluated:
|
||||
s += "evaluated"
|
||||
if th.invalid:
|
||||
s += ", invalid"
|
||||
else:
|
||||
s += " = " + juiValueString(th.result)
|
||||
else:
|
||||
s += "unevaluated"
|
||||
s += "){ " + str(th.ast) + " }"
|
||||
return s
|
||||
case PartialRecordSnapshot():
|
||||
return "<A PRS AAAAAAH>"
|
||||
case _:
|
||||
raise NotImplementedError
|
||||
|
||||
class JuiTypeError(Exception):
|
||||
pass
|
||||
|
||||
class JuiNameError(Exception):
|
||||
pass
|
||||
|
||||
class JuiStaticError(Exception):
|
||||
pass
|
||||
|
||||
class JuiRuntimeError(Exception):
|
||||
pass
|
||||
|
||||
# TODO: Better diagnostics for type errors
|
||||
def requireType(v: JuiValue, predicate: Callable[[JuiValue], bool]):
|
||||
if not predicate(v):
|
||||
raise JuiTypeError(f"type error: got {v}, needed {predicate}")
|
||||
|
||||
def requireSameType(vs: list[JuiValue], predicate: Callable[[JuiValue], bool]):
|
||||
if len(vs) == 0:
|
||||
return
|
||||
if len(set(type(v) for v in vs)) != 1:
|
||||
vals = ", ".join(str(v) for v in vs)
|
||||
raise JuiTypeError(f"type error: heterogeneous types for {vals}")
|
||||
|
||||
if not predicate(vs[0]):
|
||||
raise JuiTypeError(f"type error: got {vs[0]}, needed {predicate}")
|
||||
|
||||
# Evaluation context. This tracks the current scope and provides evaluation
|
||||
# functions, error tracking, etc.
|
||||
class Context:
|
||||
# Current top-level scope. Can be None. This is the only scope in which we
|
||||
# can add definitions.
|
||||
currentScope: MutableScopeData
|
||||
# Current closure defining the scopes that surround `currentScope`.
|
||||
currentClosure: Closure | None
|
||||
|
||||
def __init__(self, initialClosure = None):
|
||||
self.currentScope = MutableScopeData()
|
||||
self.currentClosure = initialClosure
|
||||
self._contextStack = []
|
||||
|
||||
#=== Context switch commodity ===#
|
||||
|
||||
_contextStack: list[Tuple[MutableScopeData, Closure]]
|
||||
|
||||
class ContextSwitchContextManager:
|
||||
def __init__(self, parent: "Context", c: Closure):
|
||||
self.parent = parent
|
||||
self.targetClosure = c
|
||||
|
||||
def __enter__(self):
|
||||
p = self.parent
|
||||
p._contextStack.append((p.currentScope, p.currentClosure))
|
||||
p.currentScope = MutableScopeData()
|
||||
p.currentClosure = self.targetClosure
|
||||
|
||||
def __exit__(self, exc_type, exc_value, traceback):
|
||||
p = self.parent
|
||||
s, c = p._contextStack.pop()
|
||||
p.currentScope = s
|
||||
p.currentClosure = c
|
||||
|
||||
# Temporarily switch the execution context. This is used when we need e.g.
|
||||
# to evaluate a thunk in its associated closure. Morally one could
|
||||
# construct a different Context object, but it's easier to accumulate error
|
||||
# messages and other things in the same Context object.
|
||||
# Example:
|
||||
# with self.contextSwitchAndPushNewScope(myClosure):
|
||||
# myResult = self.eval(myExprNode)
|
||||
def contextSwitchAndPushNewScope(self, c: Closure):
|
||||
return Context.ContextSwitchContextManager(self, c)
|
||||
|
||||
# Freeze the current scope and push a new empty one to start working in.
|
||||
# Example:
|
||||
# with self.pushNewScope():
|
||||
# self.addDefinition(myParamName, myParamValue)
|
||||
# myResult = self.eval(myFunctionBody)
|
||||
def pushNewScope(self):
|
||||
return self.contextSwitchAndPushNewScope(self.currentStateClosure())
|
||||
|
||||
#=== State management ===#
|
||||
|
||||
# Build a the closure of the current state
|
||||
def currentStateClosure(self):
|
||||
return Closure(parent=self.currentClosure, scope=self.currentScope)
|
||||
|
||||
# Add a new definition in the innermost scope
|
||||
def addDefinition(self, name: str, content: JuiValue | Thunk):
|
||||
return self.currentScope.addDefinition(name, content)
|
||||
|
||||
# Lookup a name
|
||||
def lookup(self, name: str) -> JuiValue | Thunk | None:
|
||||
return self.currentStateClosure().lookup(name)
|
||||
|
||||
#=== Evaluation helpers ===#
|
||||
|
||||
def makeFunction(self, params: list[Tuple[str, bool]], body: Node) \
|
||||
-> Function:
|
||||
# Check validity of the variadic parameter specification
|
||||
variadics = [i for i, (_, v) in enumerate(params) if v]
|
||||
|
||||
for (name, _) in params:
|
||||
count = len([0 for (n, _) in params if n == name])
|
||||
if count > 1:
|
||||
raise JuiStaticError(f"duplicate argument {name}")
|
||||
|
||||
if len(variadics) > 1:
|
||||
names = ", ".join(params[i][0] for i in variadics)
|
||||
raise JuiStaticError(f"multiple variadic arguments: {names}")
|
||||
|
||||
if variadics and variadics != [len(params) - 1]:
|
||||
name = params[variadic][0]
|
||||
raise JuiStaticError(f"variadic argument {name} not at the end")
|
||||
variadic = params[variadics[0]][0] if variadics else None
|
||||
|
||||
# Build function object
|
||||
return Function(closure = self.currentStateClosure(),
|
||||
body = body,
|
||||
params = [name for (name, v) in params if not v],
|
||||
variadic = variadic)
|
||||
|
||||
#=== Main evaluation functions ===#
|
||||
|
||||
# Expression evaluator; this function is pure. It can cause thunks to be
|
||||
# evaluated and thus reveal previously-undiscovered cyclic dependencies,
|
||||
# but it doesn't modify the context.
|
||||
def evalExpr(self, node: Node) -> JuiValue:
|
||||
match node.ctor, node.args:
|
||||
case Node.T.LIT, [x]:
|
||||
return x
|
||||
|
||||
case Node.T.IDENT, [name]:
|
||||
match self.lookup(name):
|
||||
case None:
|
||||
raise JuiNameError(f"name {name} not defined")
|
||||
case Thunk() as t:
|
||||
return self.evalThunk(t)
|
||||
case value:
|
||||
return value
|
||||
|
||||
case Node.T.OP, ["+", X, Y]:
|
||||
x = self.evalExpr(X)
|
||||
y = self.evalExpr(Y)
|
||||
requireSameType([x, y], juiIsAddable)
|
||||
return x + y
|
||||
|
||||
# TODO: Arithmetic: integer division, type conversions
|
||||
case Node.T.OP, [("+" | "-" | "*" | "/" | "%") as op, X, Y]:
|
||||
x = self.evalExpr(X)
|
||||
y = self.evalExpr(Y)
|
||||
requireSameType([x, y], juiIsArith)
|
||||
match op:
|
||||
case "+": return x + y
|
||||
case "-": return x - y
|
||||
case "*": return x * y
|
||||
case "/": return x / y if type(x) == float else x // y
|
||||
case "%": return x % y
|
||||
|
||||
case Node.T.OP, [("!" | "+" | "-") as op, X]:
|
||||
x = self.evalExpr(X)
|
||||
requireType(x, juiIsArith)
|
||||
match op:
|
||||
case "!": return not x
|
||||
case "+": return +x
|
||||
case "-": return -x
|
||||
|
||||
case Node.T.OP, [(">"|">="|"<"|"<="|"=="|"!=") as op, X, Y]:
|
||||
x = self.evalExpr(X)
|
||||
y = self.evalExpr(Y)
|
||||
requireSameType([x, y], juiIsComparable)
|
||||
match op:
|
||||
case ">": return x > y
|
||||
case "<": return x < y
|
||||
case ">=": return x >= y
|
||||
case "<=": return x <= y
|
||||
case "==": return x == y
|
||||
case "!=": return x != y
|
||||
|
||||
case Node.T.OP, [("&&" | "||") as op, X, Y]:
|
||||
x = self.evalExpr(X)
|
||||
y = self.evalExpr(Y)
|
||||
requireSameType([x, y], juiIsLogical)
|
||||
match op:
|
||||
case "&&": return x and y
|
||||
case "||": return x or y
|
||||
|
||||
case Node.T.OP, ["...", X]:
|
||||
x = self.evalExpr(X)
|
||||
requireType(x, juiIsUnpackable)
|
||||
raise NotImplementedError("unpack operator o(x_x)o")
|
||||
|
||||
case Node.T.OP, ["|", X, F]:
|
||||
f = self.evalExpr(F)
|
||||
requireType(f, juiIsCallable)
|
||||
return self.evalCall(f, [X])
|
||||
|
||||
case Node.T.OP, ["<|", F, X]:
|
||||
f = self.evalExpr(F)
|
||||
requireType(f, juiIsCallable)
|
||||
return self.evalCall(f, [X])
|
||||
|
||||
case Node.T.THIS, []:
|
||||
raise NotImplementedError
|
||||
|
||||
case Node.T.PROJ, []:
|
||||
raise NotImplementedError
|
||||
|
||||
case Node.T.CALL, [F, *A]:
|
||||
f = self.evalExpr(F)
|
||||
requireType(f, juiIsCallable)
|
||||
return self.evalCall(f, A)
|
||||
|
||||
case Node.T.IF, [C, T, E]:
|
||||
c = self.evalExpr(C)
|
||||
requireType(c, juiIsLogical)
|
||||
if bool(c):
|
||||
return self.evalExpr(T)
|
||||
else:
|
||||
return None if E is None else self.evalExpr(E)
|
||||
|
||||
case Node.T.RECORD, [R, *A]:
|
||||
r = self.evalExpr(R)
|
||||
requireType(r, juiIsConstructible)
|
||||
return self.evalRecordConstructor(r, A)
|
||||
|
||||
case Node.T.REC_ATTR, _:
|
||||
raise NotImplementedError
|
||||
|
||||
case Node.T.REC_VALUE, _:
|
||||
raise NotImplementedError
|
||||
|
||||
case _, _:
|
||||
raise Exception("invalid expr o(x_x)o: " + str(node))
|
||||
|
||||
def execStmt(self, node: Node) -> JuiValue:
|
||||
match node.ctor, node.args:
|
||||
case Node.T.LET_DECL, [name, X]:
|
||||
self.addDefinition(name, self.evalExpr(X))
|
||||
|
||||
case Node.T.FUN_DECL, [name, params, body]:
|
||||
self.addDefinition(name, self.makeFunction(params, body))
|
||||
|
||||
case Node.T.REC_DECL, _:
|
||||
raise NotImplementedError
|
||||
|
||||
case Node.T.SET_STMT, _:
|
||||
raise NotImplementedError
|
||||
|
||||
case _, _:
|
||||
return self.evalExpr(node)
|
||||
|
||||
# TODO: Context.eval*: continue failed computations to find other errors?
|
||||
def evalThunk(self, th: Thunk) -> JuiValue:
|
||||
if th.evaluated:
|
||||
if th.invalid:
|
||||
raise JuiRuntimeError("cyclic dependency result encountered")
|
||||
return th.result
|
||||
if th._running:
|
||||
raise JuiRuntimeError("cyclic dependency detected!")
|
||||
|
||||
with self.contextSwitchAndPushNewScope(th.closure):
|
||||
if th.thisReference is not None:
|
||||
self.addDefinition("this", th.thisReference)
|
||||
|
||||
th._running = True
|
||||
result = self.evalExpr(th.ast)
|
||||
th._running = False
|
||||
|
||||
# TODO: Set invalid = True if we continue failed computations
|
||||
th.evaluated = True
|
||||
th.result = result
|
||||
return result
|
||||
|
||||
def evalCall(self, f: JuiValue, args: list[Node]) -> JuiValue:
|
||||
# Built-in functions: just evaluate arguments and go
|
||||
# TODO: Check types of built-in function calls
|
||||
if type(f) == BuiltinFunction:
|
||||
return f.func(*[self.evalExpr(a) for a in args])
|
||||
|
||||
assert type(f) == Function and "evalCall: bad type check precondition"
|
||||
|
||||
# Check number of arguments
|
||||
req = str(len(f.params)) + ("+" if f.variadic is not None else "")
|
||||
if len(args) < len(f.params):
|
||||
raise JuiRuntimeError(f"not enough args (need {req}, got {len(args)})")
|
||||
if len(args) > len(f.params) and f.variadic is None:
|
||||
raise JuiRuntimeError(f"too many args (need {req}, got {len(args)})")
|
||||
|
||||
# TODO: In order to build variadic set I need a LIST node
|
||||
if f.variadic is not None:
|
||||
raise NotImplementedError("list node for building varargs o(x_x)o")
|
||||
|
||||
# Run into the function's scope
|
||||
with self.contextSwitchAndPushNewScope(f.closure):
|
||||
for name, node in zip(f.params, args):
|
||||
th = Thunk(ast=node, closure=self.currentStateClosure())
|
||||
self.addDefinition(name, th)
|
||||
# self.currentScope.dump()
|
||||
# self.currentClosure.dump()
|
||||
return self.execStmt(f.body)
|
||||
|
||||
def evalRecordConstructor(self, ctor: JuiValue, entries: list[Node]) \
|
||||
-> JuiValue:
|
||||
|
||||
# Collect indices of fields and arguments separately; keep the order
|
||||
args = []
|
||||
attr = []
|
||||
for i, e in enumerate(entries):
|
||||
if e.ctor == Node.T.REC_ATTR:
|
||||
attr.append(i)
|
||||
elif e.ctor == Node.T.REC_VALUE:
|
||||
args.append(i)
|
||||
else:
|
||||
assert False and "record node has weird children o(x_x)o"
|
||||
|
||||
# Base record constructor: starts out with an empty record. All
|
||||
# arguments are children. Easy.
|
||||
if type(ctor) == RecordType:
|
||||
r = Record(base=ctor, attr=dict(), children=dict())
|
||||
|
||||
if len(args) > 0:
|
||||
raise JuiRuntimeError(f"arguments given to type rec ctor")
|
||||
|
||||
# Create thunks for all entries while providing them with
|
||||
# progressively more complete PRS of r.
|
||||
prs = PartialRecordSnapshot()
|
||||
|
||||
for i in attr:
|
||||
name, label, node = entries[i].args
|
||||
th = Thunk(ast=node, closure=self.currentStateClosure())
|
||||
th.thisReference = prs.copy()
|
||||
|
||||
r.attr[name] = th
|
||||
prs.fieldThunks[name] = th
|
||||
|
||||
return r
|
||||
|
||||
# TODO: Factor this with function. In fact, this should reduce to a call
|
||||
# Check number of arguments
|
||||
req = str(len(ctor.params)) + ("+" if ctor.variadic is not None else "")
|
||||
if len(args) < len(ctor.params):
|
||||
raise JuiRuntimeError(f"not enough args (need {req}, got {len(args)})")
|
||||
if len(args) > len(ctor.params) and ctor.variadic is None:
|
||||
raise JuiRuntimeError(f"too many args (need {req}, got {len(args)})")
|
||||
|
||||
# TODO: In order to build variadic set I need a LIST node
|
||||
if ctor.variadic is not None:
|
||||
raise NotImplementedError("list node for building varargs o(x_x)o")
|
||||
|
||||
# Otherwise, it's a function so it might use
|
||||
raise NotImplementedError
|
||||
|
||||
assert type(r) == Function and \
|
||||
"evalRecordConstructor: not a record type nor a function"
|
||||
|
||||
# Thunks should be created from a closure of the entire environment
|
||||
# -> This environment is always known, _except_ for
|
||||
# partially-constructed records
|
||||
# So, naturally... patch the PRS for "this" later on?
|
||||
#
|
||||
# ## In a normal function call
|
||||
#
|
||||
# Invariant: context of evaluating the expression that has the call is
|
||||
# complete. Use it for all the arguments. Done.
|
||||
#
|
||||
# ## In a record constructor with no parameters
|
||||
#
|
||||
# All attribute values have the same context when excluding "this",
|
||||
# it's just the context surrounding the record constructor, and
|
||||
# (invariant) it is complete. Use it to create all of the thunks.
|
||||
#
|
||||
# Later, iterate through the thunks in top-down order and patch the
|
||||
# "this" by constructing PRSs.
|
||||
#
|
||||
# Invariant is satisified because each thunk's closure is complete and
|
||||
# thus the context for evaluating their contents will also be
|
||||
# complete.
|
||||
#
|
||||
# ## In a record constructor with parameters
|
||||
#
|
||||
# Again, the entire context except for "this" is well-defined and
|
||||
# complete. Use it to create thunks for all record entries, inline or
|
||||
# not.
|
||||
#
|
||||
# Prepare a thunk for the call's evaluation, based on the function's
|
||||
# environment, assigning parameter-related thunks (or lists thereof) as
|
||||
# definitions for the function's arguments.
|
||||
#
|
||||
# Use the call's thunk as base for the PRS and sweep top-to-bottom
|
||||
# assigning PRSs in every record-entry thunk.
|
||||
#
|
||||
# Return a literal record with all non-parameter entries as elements.
|
||||
# Erm, remove duplicates.
|
||||
#
|
||||
# NOTE: NO WAY TO SPECIFY AN ATTRIBUTE IN A NON-STATIC WAY.
|
||||
|
||||
# - Build thunks for all entries, with one layer of "this" data
|
||||
# - Call ctor1, pass it the argument thunks
|
||||
# - Move into ctor1's scope, add new scope with arguments
|
||||
# - Build a thunk for ctor1's body, this is second layer of "this"
|
||||
# /!\ It's not per-field! It's not per-field!
|
||||
# - Go back to toplevel, add second layer of "this" to all thunks
|
||||
# - Return a PartialRecord object that has ctor1's returned thunk as
|
||||
# base and the other fields as overrides.
|
||||
#
|
||||
# ... Looks too opaque at ctor1's level.
|
||||
# ... But then again for lazy evaluation all we need to know is it's a
|
||||
# record, and then we evaluated only when we project fields. If the
|
||||
# same thing happens with <{} there should be no problem.
|
||||
# ... Having exactly two layers is suspicious. What happens when we try
|
||||
# to get a field?
|
||||
#
|
||||
# 1. Lookup the field in the PartialRecord. If it has it, it's been
|
||||
# added by the latest constructor, and we know which it is. Evaluate
|
||||
# the thunk inside.
|
||||
# 2. Otherwise, evaluate the base thunk. In the "f <| rec { ... }" it's
|
||||
# a record update. This will yield another PartialRecord. Use it.
|
||||
# 3. This should work, but we need a PartialRecord.
|
||||
#
|
||||
# When do we convert back to a full Record?
|
||||
# How about that's what Record does, and it just has accessors?
|
||||
raise NotImplementedError
|
||||
|
||||
def force(self, v: JuiValue) -> JuiValue:
|
||||
match v:
|
||||
case Record() as r:
|
||||
for a in r.attr:
|
||||
r.attr[a] = self.force(r.attr[a])
|
||||
return r
|
||||
case Thunk() as th:
|
||||
self.evalThunk(th)
|
||||
if th.evaluated:
|
||||
return self.force(th.result)
|
||||
return th
|
||||
case _:
|
||||
return v
|
101
juic/examples/ex1.jui
Normal file
101
juic/examples/ex1.jui
Normal file
|
@ -0,0 +1,101 @@
|
|||
1+2;
|
||||
/* comment */
|
||||
3;
|
||||
let zero = 0;
|
||||
zero + 2 * 20
|
||||
/* other
|
||||
comment */
|
||||
- 8
|
||||
/ 4;
|
||||
("hello" + "," + "world");
|
||||
if(2 < 4) 8;
|
||||
if(2 > 4) 8;
|
||||
if(2 > 4) 8 else "ok";
|
||||
let z = 4;
|
||||
(z + 10) * 3;
|
||||
let helloworld = "Hello, World!";
|
||||
len("xyz" + helloworld) + 1;
|
||||
print(12);
|
||||
print(42, "42", 73, "73");
|
||||
|
||||
fun myprint3(x, y, z) = print(x, y, z);
|
||||
myprint3(1, 2, 3);
|
||||
|
||||
let u = 2;
|
||||
fun test(x, t, gr) = 42 * x + z + 7;
|
||||
let v = 9;
|
||||
test(2, 3, u);
|
||||
|
||||
record {};
|
||||
record { attr: 2 + 3; };
|
||||
|
||||
record { x: 1; y: 2; z: subrecord { u: "42"; }; };
|
||||
|
||||
/*
|
||||
record {
|
||||
fullscreen: true;
|
||||
@title jlabel { title; };
|
||||
@stack jwidget {} | stack | stretch;
|
||||
|
||||
let x = 4;
|
||||
jfkeys { x };
|
||||
|
||||
fun test(x, y, ...all) = x + y + sum(all);
|
||||
};
|
||||
|
||||
fun _(fx, cg) = if(param("FX")) fx else cg;
|
||||
fun stack(elem) = elem <{ layout: stack };
|
||||
fun stretch(elem) = elem <{ stretch_x: 1; stretch_y: 1; strech_beyond_limits: false };
|
||||
fun myvbox(elem) = elem <{ layout: vbox { spacing: 1 } };
|
||||
|
||||
rec gscreen(name, keys, ...tabs) = myvbox <| jscene {
|
||||
fullscreen: true;
|
||||
@title if(name) jlabel { name };
|
||||
@stack jwidget { ...map(stretch, tabs) } | stack | stretch;
|
||||
jfkeys { keys };
|
||||
};
|
||||
|
||||
gscreen {
|
||||
@name name: "Test";
|
||||
@fs jfileselect {};
|
||||
@shell peshell {};
|
||||
};
|
||||
|
||||
set jlabel { font: &font_rogue };
|
||||
|
||||
jlabel {
|
||||
width: 80;
|
||||
margin: this.width - 2 * 20;
|
||||
};
|
||||
let hello_world_jlabel = jlabel { "Hello, World!" };
|
||||
# implicit hole
|
||||
hello_world_jlabel { width: dsize(this.text) };
|
||||
*/
|
||||
|
||||
/*
|
||||
fun f(x) = x + 2;
|
||||
label { text: "hello"; height: 4; };
|
||||
|
||||
fun my_widget(children) =
|
||||
widget {
|
||||
height: len(children) * 20;
|
||||
...children;
|
||||
};
|
||||
// closure for my_widget function
|
||||
|
||||
my_widget {
|
||||
x: this.height; // 4
|
||||
height: 8;
|
||||
// closure for the value of y
|
||||
// "this" is my_widget:14
|
||||
y: this.height; // 8
|
||||
|
||||
// closure for "children" parameter
|
||||
// "this" is my_widget:18
|
||||
$children:
|
||||
[label { "hello my height is " + str(this.height) };
|
||||
button { height: this.x } ];
|
||||
|
||||
bg: none;
|
||||
};
|
||||
*/
|
80
juic/main.py
Normal file
80
juic/main.py
Normal file
|
@ -0,0 +1,80 @@
|
|||
import getopt
|
||||
import sys
|
||||
|
||||
import juic.parser
|
||||
import juic.eval
|
||||
import juic.builtins
|
||||
|
||||
USAGE_STRING = """
|
||||
usage: juic [OPTIONS] INPUT
|
||||
|
||||
JustUI high-level description compiler.
|
||||
|
||||
Options:
|
||||
--debug=lexer Dump the lexer output and stop.
|
||||
=parser Dump the parser output and stop.
|
||||
""".strip()
|
||||
|
||||
def usage(exitcode=None):
|
||||
print(USAGE_STRING, file=sys.stderr)
|
||||
|
||||
if exitcode is not None:
|
||||
sys.exit(exitcode)
|
||||
|
||||
def main(argv):
|
||||
# Read command-line arguments
|
||||
try:
|
||||
opts, args = getopt.gnu_getopt(argv[1:], "", ["help", "debug="])
|
||||
opts = dict(opts)
|
||||
|
||||
if len(argv) == 1 or "-h" in opts or "--help" in opts:
|
||||
usage(0)
|
||||
if len(args) < 1:
|
||||
usage(1)
|
||||
if len(args) > 1:
|
||||
raise getopt.GetoptError("only one input file can be specified")
|
||||
|
||||
except getopt.GetoptError as e:
|
||||
print("error:", e, file=sys.stderr)
|
||||
print("Try '{} --help' for details.".format(argv[0]), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
#---
|
||||
|
||||
with open(args[0], "r") as fp:
|
||||
source = fp.read()
|
||||
|
||||
try:
|
||||
lexer = juic.parser.JuiLexer(source, args[0])
|
||||
if opts.get("--debug") == "lexer":
|
||||
lexer.dump()
|
||||
return 0
|
||||
|
||||
parser = juic.parser.JuiParser(lexer)
|
||||
ast = parser.scope()
|
||||
if opts.get("--debug") == "parser":
|
||||
for e in ast.args:
|
||||
e.dump()
|
||||
print("---")
|
||||
return 0
|
||||
|
||||
except juic.parser.SyntaxError as e:
|
||||
print(e.loc, ": ", "\x1b[31merror:\x1b[0m ", e.message, sep="",
|
||||
file=sys.stdout)
|
||||
return 1
|
||||
|
||||
ctx = juic.eval.Context(juic.builtins.builtinClosure)
|
||||
|
||||
for e in ast.args:
|
||||
e.dump()
|
||||
v = ctx.execStmt(e)
|
||||
v = ctx.force(v)
|
||||
print(">>>>>>>", juic.eval.juiValueString(v))
|
||||
|
||||
ctx.currentScope.dump()
|
||||
ctx.currentClosure.dump()
|
||||
|
||||
print("TODO: Codegen not implemented yet o(x_x)o")
|
||||
return 1
|
||||
|
||||
sys.exit(main(sys.argv))
|
494
juic/parser.py
Normal file
494
juic/parser.py
Normal file
|
@ -0,0 +1,494 @@
|
|||
from dataclasses import dataclass
|
||||
import typing
|
||||
import enum
|
||||
import sys
|
||||
import re
|
||||
|
||||
import juic.datatypes
|
||||
|
||||
@dataclass
|
||||
class Loc:
|
||||
path: str
|
||||
line: int
|
||||
column: int
|
||||
|
||||
def __str__(self):
|
||||
path = self.path or "(anonymous)"
|
||||
return f"{path}:{self.line}:{self.column}"
|
||||
|
||||
@dataclass
|
||||
class SyntaxError(Exception):
|
||||
loc: Loc
|
||||
message: str
|
||||
|
||||
@dataclass
|
||||
class Token:
|
||||
type: typing.Any
|
||||
value: typing.Any
|
||||
loc: Loc
|
||||
|
||||
def __str__(self):
|
||||
if self.value is None:
|
||||
return f"{self.type}"
|
||||
else:
|
||||
return f"{self.type}({self.value})"
|
||||
|
||||
class NaiveRegexLexer:
|
||||
"""
|
||||
Base class for a very naive regex-based lexer. This class provides the
|
||||
naive matching algorithm that applies all regexes at the current point and
|
||||
constructs a token with the longest, earliest match in the list. Regular
|
||||
expressions for tokens as specified in a class-wide TOKEN_REGEX list which
|
||||
consist of triples (regex, token type, token value).
|
||||
"""
|
||||
|
||||
# Override with list of (regex, token type, token value). Both the token
|
||||
# type and value can be functions, in which case they'll be called with the
|
||||
# match object as parameter.
|
||||
TOKEN_REGEX = []
|
||||
# Override with token predicate that matches token to be discarded and not
|
||||
# sent to the parser (typically, whitespace and comments).
|
||||
TOKEN_DISCARD = lambda _: False
|
||||
|
||||
def __init__(self, input, inputFilename):
|
||||
self.input = input
|
||||
self.inputFilename = inputFilename
|
||||
self.line = 1
|
||||
self.column = 1
|
||||
|
||||
# TODO: Precompile the regular expressions
|
||||
|
||||
def loc(self):
|
||||
return Loc(self.inputFilename, self.line, self.column)
|
||||
|
||||
def raiseError(self, message):
|
||||
raise SyntaxError(self.loc, message)
|
||||
|
||||
def advancePosition(self, lexeme):
|
||||
for c in lexeme:
|
||||
if c == "\n":
|
||||
self.line += 1
|
||||
self.column = 0
|
||||
self.column += 1
|
||||
|
||||
def nextToken(self):
|
||||
"""Return the next token in the input stream, None at EOF."""
|
||||
if not len(self.input):
|
||||
return None
|
||||
|
||||
longestMatch = None
|
||||
longestMatchIndex = -1
|
||||
|
||||
for i, (regex, _, _) in enumerate(self.TOKEN_REGEX):
|
||||
if (m := re.match(regex, self.input)):
|
||||
if longestMatch is None or len(m[0]) > len(longestMatch[0]):
|
||||
longestMatch = m
|
||||
longestMatchIndex = i
|
||||
|
||||
if longestMatch is None:
|
||||
nextWord = self.input.split(None, 1)[0]
|
||||
self.raiseError(f"unknown syntax '{nextWord}'")
|
||||
|
||||
# Build the token
|
||||
_, type_info, value_info = self.TOKEN_REGEX[longestMatchIndex]
|
||||
m = longestMatch
|
||||
|
||||
typ = type_info(m) if callable(type_info) else type_info
|
||||
value = value_info(m) if callable(value_info) else value_info
|
||||
t = Token(typ, value, self.loc())
|
||||
|
||||
self.advancePosition(m[0])
|
||||
# Urgh. I need to find how to match a regex at a specific offset.
|
||||
self.input = self.input[len(m[0]):]
|
||||
return t
|
||||
|
||||
def lex(self):
|
||||
"""Return the next token that's visible to the parser, None at EOF."""
|
||||
t = self.nextToken()
|
||||
discard = type(self).TOKEN_DISCARD
|
||||
while t is not None and discard(t):
|
||||
t = self.nextToken()
|
||||
return t
|
||||
|
||||
def dump(self, showDiscarded=False, fp=sys.stdout):
|
||||
"""Dump all remaining tokens on a stream, for debugging."""
|
||||
t = 0
|
||||
discard = type(self).TOKEN_DISCARD
|
||||
while t is not None:
|
||||
t = self.nextToken()
|
||||
if t is not None and discard(t):
|
||||
if showDiscarded:
|
||||
print(t, "(discarded)")
|
||||
else:
|
||||
print(t)
|
||||
|
||||
class LL1Parser:
|
||||
"""
|
||||
Base class for an LL(1) recursive descent parser. This class provides the
|
||||
base mechanisms for hooking up a lexer, consuming tokens, checking the
|
||||
lookahead, and combinators for writing common types of rules such as
|
||||
expressions with operator precedence.
|
||||
"""
|
||||
|
||||
def __init__(self, lexer):
|
||||
self.lexer = lexer
|
||||
self.la = None
|
||||
self.advance()
|
||||
|
||||
def advance(self):
|
||||
"""Return the next token and update the lookahead."""
|
||||
t, self.la = self.la, self.lexer.lex()
|
||||
return t
|
||||
|
||||
def atEnd(self):
|
||||
return self.la is None
|
||||
|
||||
def raiseErrorAt(self, token, message):
|
||||
raise SyntaxError(token.loc, message)
|
||||
|
||||
def expect(self, types, pred=None, optional=False):
|
||||
"""
|
||||
Read the next token, ensuring it is one of the specified types; if
|
||||
`pred` is specified, also tests the predicate. If `optional` is set,
|
||||
returns None in case of mismatch rather than raising an error.
|
||||
"""
|
||||
|
||||
if not isinstance(types, list):
|
||||
types = [types]
|
||||
if self.la is not None and self.la.type in types and \
|
||||
(pred is None or pred(self.la)):
|
||||
return self.advance()
|
||||
if optional:
|
||||
return None
|
||||
|
||||
expected = ", ".join(str(t) for t in types)
|
||||
err = f"expected one of {expected}, got {self.la}"
|
||||
if pred is not None:
|
||||
err += " (with predicate)"
|
||||
self.raiseErrorAt(self.la, err)
|
||||
|
||||
# Rule combinators implementing unary and binary operators with precedence
|
||||
|
||||
def binaryOpsLeft(ctor, ops):
|
||||
def decorate(f):
|
||||
def symbol(self):
|
||||
e = f(self)
|
||||
while (op := self.expect(ops, optional=True)) is not None:
|
||||
e = ctor(op, [e, f(self)])
|
||||
return e
|
||||
return symbol
|
||||
return decorate
|
||||
|
||||
def binaryOps(ctor, ops, *, rassoc=False):
|
||||
def decorate(f):
|
||||
def symbol(self):
|
||||
lhs = f(self)
|
||||
if (op := self.expect(ops, optional=True)) is not None:
|
||||
rhs = symbol(self) if rassoc else f(self)
|
||||
return ctor(op, [lhs, rhs])
|
||||
else:
|
||||
return lhs
|
||||
return symbol
|
||||
return decorate
|
||||
|
||||
def binaryOpsRight(ctor, ops):
|
||||
return LL1Parser.binaryOps(ctor, ops, rassoc=True)
|
||||
|
||||
def unaryOps(ctor, ops, assoc=True):
|
||||
def decorate(f):
|
||||
def symbol(self):
|
||||
if (op := self.expect(ops, optional=True)) is not None:
|
||||
arg = symbol(self) if assoc else f(self)
|
||||
return ctor(op, [arg])
|
||||
else:
|
||||
return f(self)
|
||||
return symbol
|
||||
return decorate
|
||||
|
||||
#---
|
||||
|
||||
def unescape(s: str) -> str:
|
||||
return s.encode("raw_unicode_escape").decode("unicode_escape")
|
||||
|
||||
class JuiLexer(NaiveRegexLexer):
|
||||
T = enum.Enum("T",
|
||||
["WS", "KW", "COMMENT",
|
||||
"INT", "FLOAT", "STRING",
|
||||
"IDENT", "ATTR", "VAR", "LABEL", "FIELD", "CXXIDENT"])
|
||||
|
||||
RE_COMMENT = r'(#|//)[^\n]*|/\*([^/]|/[^*])+\*/'
|
||||
RE_INT = r'0|[1-9][0-9]*|0b[0-1]+|0o[0-7]+|0[xX][0-9a-fA-F]+'
|
||||
# RE_FLOAT = r'([0-9]*\.[0-9]+|[0-9]+\.[0-9]*|[0-9]+)([eE][+-]?{INT})?f?'
|
||||
|
||||
RE_KW = r'\b(else|fun|if|let|rec|set|this)\b'
|
||||
RE_IDENT = r'[\w_][\w0-9_]*'
|
||||
RE_ATTR = r'({})\s*(?:@({}))?\s*:'.format(RE_IDENT, RE_IDENT)
|
||||
RE_VAR = r'\$(\.)?' + RE_IDENT
|
||||
RE_LABEL = r'@' + RE_IDENT
|
||||
RE_FIELD = r'\.' + RE_IDENT
|
||||
RE_CXXIDENT = r'&(::)?[a-zA-Z_]((::)?[a-zA-Z0-9_])*'
|
||||
|
||||
RE_STRING = r'["]((?:[^\\"]|\\"|\\n|\\t|\\\\)*)["]'
|
||||
RE_PUNCT = r'\.\.\.|[.,:;=(){}]'
|
||||
# TODO: Extend operator language to allow custom operators?
|
||||
RE_OP = r'<\||>=|<=|!=|==|\|\||&&|<{|[|+*/%-<>!]'
|
||||
|
||||
TOKEN_REGEX = [
|
||||
(r'[ \t\n]+', T.WS, None),
|
||||
(RE_COMMENT, T.COMMENT, None),
|
||||
(RE_INT, T.INT, lambda m: int(m[0], 0)),
|
||||
# FLOAT
|
||||
|
||||
(RE_KW, T.KW, lambda m: m[0]),
|
||||
(RE_IDENT, T.IDENT, lambda m: m[0]),
|
||||
(RE_ATTR, T.ATTR, lambda m: (m[1], m[2])),
|
||||
(RE_VAR, T.VAR, lambda m: m[0][1:]),
|
||||
(RE_LABEL, T.LABEL, lambda m: m[0][1:]),
|
||||
(RE_FIELD, T.FIELD, lambda m: m[0][1:]),
|
||||
(RE_CXXIDENT, T.CXXIDENT, lambda m: m[0][1:]),
|
||||
|
||||
(RE_STRING, T.STRING, lambda m: unescape(m[1])),
|
||||
(RE_PUNCT, lambda m: m[0], None),
|
||||
(RE_OP, lambda m: m[0], None),
|
||||
]
|
||||
TOKEN_DISCARD = lambda t: t.type in [JuiLexer.T.WS, JuiLexer.T.COMMENT]
|
||||
|
||||
@dataclass
|
||||
class Node:
|
||||
T = enum.Enum("T", [
|
||||
"LIT", "IDENT", "OP", "THIS", "PROJ", "CALL", "IF", "SCOPE",
|
||||
"RECORD", "REC_ATTR", "REC_VALUE",
|
||||
"LET_DECL", "FUN_DECL", "REC_DECL", "SET_STMT"])
|
||||
ctor: T
|
||||
args: list[typing.Any]
|
||||
|
||||
def dump(self, indent=0):
|
||||
print(" " * indent + self.ctor.name, end=" ")
|
||||
match self.ctor, self.args:
|
||||
case Node.T.LIT, [v]:
|
||||
print(repr(v))
|
||||
case Node.T.IDENT, [v]:
|
||||
print(v)
|
||||
case Node.T.OP, [op, *args]:
|
||||
print(op)
|
||||
self.dumpArgs(args, indent)
|
||||
case _, args:
|
||||
print("")
|
||||
self.dumpArgs(args, indent)
|
||||
|
||||
def dumpArgs(self, args, indent=0):
|
||||
for arg in args:
|
||||
if isinstance(arg, Node):
|
||||
arg.dump(indent + 1)
|
||||
else:
|
||||
print(" " * (indent + 1) + str(arg))
|
||||
|
||||
def __str__(self):
|
||||
match self.ctor, self.args:
|
||||
case Node.T.LIT, [v]:
|
||||
return repr(v)
|
||||
case Node.T.IDENT, [v]:
|
||||
return v
|
||||
case ctor, args:
|
||||
return f"{ctor.name}({', '.join(str(a) for a in args)})"
|
||||
|
||||
def mkOpNode(op, args):
|
||||
return Node(Node.T.OP, [op.type] + args)
|
||||
|
||||
# TODO: Parser: Track locations when building up AST nodes
|
||||
class JuiParser(LL1Parser):
|
||||
|
||||
def expectKeyword(self, *args):
|
||||
return self.expect(JuiLexer.T.KW, pred=lambda t: t.value in args).value
|
||||
|
||||
# A list of elementFunction separated by sep, with an optional final sep.
|
||||
# There must be a distinguishable termination marker "term" in order to
|
||||
# detemrine whether there are more elements incoming. "term" can either be
|
||||
# a token type or a callable applied to self.la.
|
||||
def separatedList(self, elementFunction, *, sep, term):
|
||||
elements = []
|
||||
termFunction = term if callable(term) else lambda la: la.type == term
|
||||
while not termFunction(self.la):
|
||||
elements.append(elementFunction())
|
||||
if termFunction(self.la):
|
||||
break
|
||||
self.expect(sep)
|
||||
return elements
|
||||
|
||||
# expr0 ::= "null" | "true" | "false" (constants)
|
||||
# | INT | FLOAT | STRING | CXXIDENT (literals)
|
||||
# | IDENT
|
||||
# | "(" expr ")"
|
||||
# | "{" scope_stmt,* "}"
|
||||
def expr0(self):
|
||||
T = JuiLexer.T
|
||||
lit_kws = ["this", "null", "true", "false"]
|
||||
t = self.expect(
|
||||
[T.INT, T.FLOAT, T.STRING, T.IDENT, T.CXXIDENT, T.KW, "("],
|
||||
pred = lambda t: t.type != T.KW or t.value in lit_kws)
|
||||
|
||||
match t.type:
|
||||
case T.INT | T.FLOAT | T.STRING:
|
||||
node = Node(Node.T.LIT, [t.value])
|
||||
case T.CXXIDENT:
|
||||
node = Node(Node.T.LIT, [juic.datatypes.CXXQualid(t.value)])
|
||||
case T.IDENT:
|
||||
node = Node(Node.T.IDENT, [t.value])
|
||||
case T.KW if t.value == "this":
|
||||
node = Node(Node.T.THIS, [])
|
||||
case T.KW if t.value == "null":
|
||||
node = Node(Node.T.LIT, [None])
|
||||
case T.KW if t.value in ["true", "false"]:
|
||||
node = Node(Node.T.LIT, t.value == "true")
|
||||
case "(":
|
||||
node = self.expr()
|
||||
self.expect(")")
|
||||
return node
|
||||
|
||||
# The following are in loose -> tight precedence order:
|
||||
# expr1 ::= expr1 <binary operator> expr1
|
||||
# | <unary operator> expr1
|
||||
# | expr0 "{" record_entry,* "}" (record construction)
|
||||
# | expr0 "<{" record_entry,* "}" (record update)
|
||||
# | expr0 "(" expr,* ")" (function call)
|
||||
# | expr0 "." ident (projection, same prec as call)
|
||||
@LL1Parser.binaryOpsRight(mkOpNode, ["|"])
|
||||
@LL1Parser.binaryOpsLeft(mkOpNode, ["<|"])
|
||||
@LL1Parser.binaryOpsLeft(mkOpNode, ["||"])
|
||||
@LL1Parser.binaryOpsLeft(mkOpNode, ["&&"])
|
||||
@LL1Parser.binaryOps(mkOpNode, [">", ">=", "<", "<=", "==", "!="])
|
||||
@LL1Parser.binaryOpsLeft(mkOpNode, ["+", "-"])
|
||||
@LL1Parser.binaryOpsLeft(mkOpNode, ["*", "/", "%"])
|
||||
@LL1Parser.unaryOps(mkOpNode, ["!", "+", "-", "..."])
|
||||
def expr1(self):
|
||||
node = self.expr0()
|
||||
|
||||
# Tight postfix operators
|
||||
while (t := self.expect([JuiLexer.T.FIELD, "("], optional=True)) \
|
||||
is not None:
|
||||
match t.type:
|
||||
case JuiLexer.T.FIELD:
|
||||
node = Node(Node.T.PROJ, [node, t.value])
|
||||
case "(":
|
||||
args = self.separatedList(self.expr, sep=",", term=")")
|
||||
self.expect(")")
|
||||
node = Node(Node.T.CALL, [node, *args])
|
||||
|
||||
# Postfix update or record creation operation
|
||||
while self.la.type in ["{", "<{"]:
|
||||
entries = self.record_literal()
|
||||
node = Node(Node.T.RECORD, [node, *entries])
|
||||
|
||||
return node
|
||||
|
||||
# expr2 ::= expr1
|
||||
# | "if" "(" expr ")" expr1 ("else" expr2)?
|
||||
def expr2(self):
|
||||
match self.la.type, self.la.value:
|
||||
case JuiLexer.T.KW, "if":
|
||||
self.expectKeyword("if")
|
||||
self.expect("(")
|
||||
cond = self.expr()
|
||||
self.expect(")")
|
||||
body1 = self.expr1()
|
||||
|
||||
if self.la.type == JuiLexer.T.KW and self.la.value == "else":
|
||||
self.expectKeyword("else")
|
||||
body2 = self.expr2()
|
||||
else:
|
||||
body2 = None
|
||||
|
||||
return Node(Node.T.IF, [cond, body1, body2])
|
||||
case _, _:
|
||||
return self.expr1()
|
||||
|
||||
def expr(self):
|
||||
return self.expr2()
|
||||
|
||||
# record_literal ::= "{" record_entry,* "}"
|
||||
# record_entry ::= LABEL? ATTR? expr
|
||||
# | let_decl
|
||||
# | fun_rec_decl
|
||||
# | set_stmt
|
||||
def record_literal(self):
|
||||
# TODO: Distinguish constructor and update
|
||||
self.expect(["{", "<{"])
|
||||
entries = self.separatedList(self.record_entry, sep=";", term="}")
|
||||
self.expect("}")
|
||||
return entries
|
||||
|
||||
def record_entry(self):
|
||||
T = JuiLexer.T
|
||||
label_t = self.expect(T.LABEL, optional=True)
|
||||
label = label_t.value if label_t is not None else None
|
||||
|
||||
match self.la.type, self.la.value:
|
||||
case T.ATTR, _:
|
||||
t = self.expect(T.ATTR)
|
||||
e = self.expr()
|
||||
return Node(Node.T.REC_ATTR, [t.value[0], label, e])
|
||||
case T.KW, "let":
|
||||
if label is not None:
|
||||
self.raiseErrorAt(label_t, "label not allowed with let")
|
||||
return self.let_decl()
|
||||
case T.KW, ("fun" | "rec"):
|
||||
if label is not None:
|
||||
self.raiseErrorAt(label_t, "label not allowed with fun/rec")
|
||||
return self.fun_rec_decl()
|
||||
case T.KW, "set":
|
||||
if label is not None:
|
||||
self.raiseErrorAt(label_t, "label not allowed with set")
|
||||
return self.set_stmt()
|
||||
case _, _:
|
||||
return Node(Node.T.REC_VALUE, [self.expr()])
|
||||
|
||||
# let_decl ::= "let" ident "=" expr
|
||||
def let_decl(self):
|
||||
self.expectKeyword("let")
|
||||
ident = self.expect(JuiLexer.T.IDENT).value
|
||||
self.expect("=")
|
||||
expr = self.expr()
|
||||
return Node(Node.T.LET_DECL, [ident, expr])
|
||||
|
||||
# fun_rec_decl ::= ("fun" | "rec") ident "(" fun_rec_param,* ")" "=" expr
|
||||
# fun_rec_param ::= "..."? ident
|
||||
def fun_rec_param(self):
|
||||
variadic = self.expect("...", optional=True) is not None
|
||||
ident = self.expect(JuiLexer.T.IDENT).value
|
||||
return (ident, variadic)
|
||||
|
||||
def fun_rec_decl(self):
|
||||
t = self.expectKeyword("fun", "rec")
|
||||
ident = self.expect(JuiLexer.T.IDENT).value
|
||||
self.expect("(")
|
||||
params = self.separatedList(self.fun_rec_param, sep=",", term=")")
|
||||
self.expect(")")
|
||||
self.expect("=")
|
||||
body = self.expr()
|
||||
return Node(Node.T.FUN_DECL if t == "fun" else Node.T.REC_DECL,
|
||||
[ident, params, body])
|
||||
|
||||
# TODO: Check variadic param validity
|
||||
|
||||
# set_stmt ::= "set" ident record_literal
|
||||
def set_stmt(self):
|
||||
self.expectKeyword("set")
|
||||
ident = self.expect(JuiLexer.T.IDENT)
|
||||
entries = self.record_literal()
|
||||
return Node(Node.T.SET_STMT, [ident, *entries])
|
||||
|
||||
def scope(self):
|
||||
isNone = lambda t: t is None
|
||||
entries = self.separatedList(self.scope_stmt, sep=";", term=isNone)
|
||||
return Node(Node.T.SCOPE, entries)
|
||||
|
||||
def scope_stmt(self):
|
||||
match self.la.type, self.la.value:
|
||||
case JuiLexer.T.KW, "let":
|
||||
return self.let_decl()
|
||||
case JuiLexer.T.KW, ("fun" | "rec"):
|
||||
return self.fun_rec_decl()
|
||||
case JuiLexer.T.KW, "set":
|
||||
return self.set_stmt()
|
||||
case _:
|
||||
return self.expr()
|
Loading…
Reference in a new issue