This page is intended for developers of the backend MethodScript engine, not
for assistance with writing scripts in MethodScript. Before beginning development
with MethodScript, you should have a firm grasp of Java, including concepts
like inheritance, abstraction, reflection, annotations, generics, etc. Additionally,
you should have a firm grasp of how the language itself works, from the scripting
perspective.

== Abstraction ==

In general, the compiler works under the assumption that the end runtime is unknown. When compilation
is initialized, only a few key functions are strictly required to be "core" functions. That is,
they MUST have the same implementation in all implementations. Once the Abstract Syntax
Tree (AST) is created in memory, it is then linked to a particular platform, which can then
be either used as an in-memory binary, or can be cross compiled to something else.

The commonality between all implementations of MethodScript is limited to the
core functions including procedure related functions, exception related functions,
control flow related functions, basic logic and operator functions, object creation
related functions, and data handling functions. All implementations must accept
that the compiler will use the default implementation of all these functions during
compilation, for static code analysis. Most of these functions are deeply integrated
with the compiler anyways, and cannot be generically separated from the compilation
process regardless.

== Compilation ==
There are 3 distinct stages during compilation. Lexing, Compiling, and Optimizing/Linking.

=== Lexing ===
Lexing is a standard process that converts the source code into a token stream. This
process follows fairly standard algorithms, and uses a generic mechanism to add new
tokens easily. This mechanism could potentially be dynamically expanded per platform,
should the need arise.

=== Compiling ===
Compilation converts the token stream into a Abstract Syntax Tree (AST). This step
can emit only a few errors, (mostly mismatched parenthesis/braces) because the
lexer will have already halted if the syntax is wrong, and the optimizer is what
actually causes linking errors. The compiler is a typical recursive decent parser,
only it offloads the complexity of the infix notation parsing to the __autoconcat__
function, which runs separately later, during optimization.

=== Optimizing/Linking ===

Once initial code compilation has occurred, linking happens. Since
the linking process happens after control structures have been analysed, this allows
for "meta programming" using existing control structures familiar to the coder.
In addition to the ${} compiler directive statements, this allows for meta programming
to be accomplished much in the same way C++ provides #define, #ifdef, and other
preprocessor directives, but there is no way for the user to go beyond the intended
cases of simple function substitution, for instance, conditionally #ifdefing out
a closing bracket. This is because this substitution occurs AFTER the lexer runs, not
before it, as happens in C++.

Optimization of the core functions occurs before linking, so actually this step
can be thought of as optimization/linking/optimization. By optimizing the core
functions however, we generally only ''remove'' code from the AST. Some data
transformations do occur, but most of the core functions do not "run" at that time.
Once the initial optimization happens, we link to the actual runtime requested,
by passing control to the individual functions that are linked to that particular
runtime. In combination with the environment, the platform is able to perform whatever
actions it needs to complete the process with the generic compiler, while still
custom tailoring the output to suit that particular platform.

Individual functions have the opportunity to optimize themselves, either by running during
compile time (should they have the ability to) and therefore completely removing themselves
from the AST, or by simply emitting warnings/errors for whatever conditions they can
check for.

== Adding a function ==
Adding a function is simple. Create a new class that implements
%%GET_SIMPLE_CLASS|.*|Function%%, (or more probably extends %%GET_SIMPLE_CLASS|.*|AbstractFunction%%
and tag it with @api. It is important to read and be very familiar with the methods
in %%GET_SIMPLE_CLASS|.*|Function%%, so you know which methods that are optional should
be overridden.

=== Using the new argument builder ===
The new argument builder greatly simplifies argument parsing, by providing generic type safety, range, and other
compile time and runtime checks. This also provides the advantage of putting the documentation right
next to the elements, and creating more documentation from the elements themselves, instead of relying on the
programmer to provide the data correctly. This makes it so that if the documentation doesn't list a restriction,
for instance, but the restriction being missing is a bug, then the bug will be in both the documentation AND the
code, and will therefore be much easier to catch. Besides restrictions, which are discussed below, the biggest thing
this provides is type safety. Until strong typing is fully supported, the system will be mostly a runtime-only check,
though hard coded values that are incorrect will be able to be caught at runtime. Generics are also supported,
and are required for return types that return closures and arrays, and eventually
all parameters that are closures or arrays. These restrictions will be caught via unit test, as Java has no way
to make this a compile error. Additionally, several restrictions (used as MAnnotations) may be added, which will provide
runtime checks on the data passed in, or will check at compile time if possible. These restrictions are built in to
the typing system, and ensure that the data passed in with be within those restrictions, whether they were checked
at runtime or compile time. Using the new argument builder itself is optional. Providing the return type and arguments
with the returnType() and arguments() methods is not optional, but using it to re-parse the data is. The way it works is
that the function's arguments list is retrieved by the typing system once, to verify the arguments, using generic algorithms.
The function itself can then use the builder to cast the data into the correct types, which is guaranteed to succeed at that point
(barring programmer error, which will cause an Error to be thrown), and the code after that can safely assert that the restrictions
placed on the parameters is true. However, if the code itself duplicates these checks, or manually casts the values, that will
continue to work, since the raw arguments are passed in to the function as is. However, new code is strongly advised to use
the builder, since eventually, backwards compatibility may be broken, and the parameters may not be passed in except as a list.
Additionally, if the code simply requires the arguments to be processed manually, it is a sign that the arguments themselves
are too complicated, and should be simplified.

=== Restrictions ===
Restrictions work like parameter annotations in Java, for instance, if you had the following method signature in Java:
<pre>
public void func(String s, @Annotation String y){

}
</pre>

The parameter "y" is tagged with the @Annotation tag, which may provide meta information to some reflective library. In MethodScript,
the MAnnotations available in Java will eventually be available to scripts, but in the meantime, they are only available to built in
functions. Regardless, the parameter restrictions will all work the same, they work to reduce the effort required to checking rote
aspects of the parameter, and provide a way for the runtime to generically handle those cases. Additionally, the compiler is aware of
many of these annotations, and where possible, will provide the same functionality, but at compile time, where possible, allowing errors
to be caught more quickly.

The annotations that are supported by either the compiler or runtime are listed below, but all of them must implement
GET_SIMPLE_CLASS|.*compiler|CompilerAwareAnnotation to be valid parameter annotations. Each annotation has it's own
documentation in the normal API.

==== @{NonNull} ====
Parameters tagged with this cannot be assigned null.

==== @{Ranged} ====
Parameters tagged with this must be between a given range. There is the option of setting the limits
to be either inclusive or exclusive.

==== @{FormatString} ====
Parameters tagged with this must match a regex. It is only taggable on CStrings.

=== Documentation and argument builder ===
Each function has embedded java doc and type information. This is extremely important
to keep accurate, both for the sake of the user, and for a technical need. The docs()
method should return the plain text user readable documentation, which summarizes the
behavior of the function. This information isn't used programmatically in any way by
the compiler, but it is used of course by the users, and should accurately reflect
the behavior of the function. Previously, the docs() method needed to return a string
in the format <code>returnType {argumentList} documentation</code>, however, this is
no longer necessary, since the return type and argument list are used programmatically
elsewhere, and therefore specified in different methods.

The returnType() method should return an %%GET_SIMPLE_CLASS|.*arguments|Argument%%, using
the "nameless" constructor of the Argument class. This simply provides the documentation
and return type of the function, no "name" is needed. There is a pre-constructed Argument.VOID
object that should be used in the case where the function returns void. Additionally, there
are a few low level functions that cause execution to terminate in a non standard way, for instance,
the die() method, or return(). These must use the Argument.NONE member, and should also likely
provide the %%GET_SIMPLE_CLASS|.*Optimizable|OptimizationOption%%.TERMINAL optimization.

The arguments() method returns the function's signature. For some functions, this will be quite
complex, for others, it should be straightforward. By design, no arguments are passed to this
method, because all arguments (and their defaults) should be constant, and should never vary
based on runtime parameters. (This complicates the design, but more importantly prevents
certain optimizations from occurring.) Functions may have multiple signatures, that is,
completely conflicting signature types, though this behavior is only provided for backwards
compatibility, and should not be used for new functions. Disjoint types are recommended instead,
or general simplification of the function instead. See the methods in the
%%GET_SIMPLE_CLASS|.*arguments|Argument%% class for more information on the various
options available when creating arguments, and %%GET_SIMPLE_CLASS|.*|ArgumentBuilder%%
for information about the ArgumentBuilder as a whole.

== Adding an event ==
Adding an event is only slightly more complicated than adding a function, though
the addition of platform specific abstraction layers may complicate the process
some, both for events and functions. In general, adding an event only requires
two steps, though in practice it may require additions in several places. The first
step is to provide the event object via implementing %%GET_SIMPLE_CLASS|.*|Event%%
(or extending %%GET_SIMPLE_CLASS|.*|AbstractGenericEvent%%) and tagging it with @api. The
second step is to actually hook into whatever system there is for actually triggering
the events, and calling %%GET_SIMPLE_CLASS|.*|EventUtils%% TriggerListener method, with
the event name and event driver, and the actual event object that it will process.
This method will then find all the user bound events, and decide if they actually
need to be triggered, then call the appropriate methods in the Event object.

The factors that can further complicate this are the fact that you will likely be using
abstract event handlers for all of the actual event objects, and that some events
are not as straightforward to modify. These pain points will hopefully be corrected
in the future.
