AuroraRuntime/readme.md

## IN DEVELOPMENT

## AuroraRuntime

The Aurora Runtime is an platform abstraction layer for cross-platform C++ development targeting<br>
embedded and PC systems. Simply fetch a binary package for your toolchain or integrate the build<br> 
scripts into your applications build pipeline to get started. <br>

View this raw file for improved formatting

## Features

- Lightweight threading and synchronization primitives
- Async threading primitives, including WaitMultipleObjects paradigm
- Asynchronous and synchronous IO abstraction 
- Optional event driven async programming paradigm
- Console; graphical and standard; binary and UTF-8 logger
- Debug and Telementry; asserts, exception logging, fio, nio backends
- Crypto ECC/[25519, P-384, P-256], [AES, RSA, X509], [common digests]
- IPC [WIP]
- Network
- Random
- Hardware Info
- FIO settings registry
- Compression
- Locale and encoding
- C++ utility templates and macros
- Follows all strings are UTF-8 convention

## Links

API: <br>
Doxygen: <br>
Examples: <br>
Tests: <br>
Cmake-stable:<br>
Build Pipeline: https://git.reece.sx/AuroraPipeline/Build

## Utilities

Aurora Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraUtils.hpp <br>
Aurora Macro Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraMacros.hpp <br>
Aurora Overloadable Type Declerations: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraTypedefs.hpp


## Logging

~~Aurora Runtime does not attempt to implement your favourite production logger. We instead <br>
implement a subscription based log message dispatcher with some default backends including <br>
a file logger, Windows debug logging, Windows conhost stdin/out using UTF-8, UNIX stdin/out <br>
respecting the applications codepage, a wxWidgets toolkit GUI, and hopefully more to come. ~~ <br>


Additionally, consoles that provide an input stream can be used in conjunction with the parse <br>
subsystem to provide basic command-based deserialization, tokenization, and dispatch of UTF-8 <br>
translated strings regardless of the system locale

## Exceptions

Through the use of compiler internal overloads, ELF hooking, and Win32 `AddVectoredExceptionHandler`, Aurora <br>
Runtime hooks exceptions at the time of throw, including *some* out of ecosystem exceptions, providing detailed <br>
telemetry of the object type, object string, and backtrace. In addition, the `AuDebug` namespace provides TLS based <br>
last-error and last-backtrace methods. <br>

EXCEPTIONS ARE NOT CONTROL FLOW...<br>
- Aurora Runtime WILL attempt to mitigate exceptions in internal logic
- Aurora Runtime WILL NOT abuse exceptions to communicate failure
- Aurora Runtime WILL try to decouple internal exceptions from the API
- Aurora Runtime WILL NOT use anything that automatically crashes on exception catch (no-nothrow)
- Aurora Runtime WILL provide extended exception information to telemetry backends and through the `AuDebug` namespace
- Aurora Runtime WILL NOT make any guarantees of being globally-nothrow; however, it should be a safe assumption in non-critical environments 

`SysPanic` can be used to format a `std::terminate`-like exit condition, complete with telemetry data and safe cleanup.


## Loop

Aurora Runtime offers a main loop that connects multiple input sources into one delegate.  <br>
Timers, semaphores, mutexes, events, X11, FDs, Win32 msg loop, macos, IPC, file aio handles, and<br>
async runner main loop sources will be supported. This equates to a cross-platfom equivalent of<br>
NT's MsgWaitForMultipleObjects in the form of a MainLoop object and a WaitMultiple function.

## Thread Primitives

The Aurora Runtime provides platform optimized threading primitives inheriting from a featureful<br>
IWaitable interface. Each method is guaranteed.


```
IWaitable
   bool TryLock()
   void Lock(relativeTimeoutInMilliseconds)
   void Lock()
   void Unlock() 
```

Included high performance primitives
- arbitrary IWaitable condition variable
- condition mutex : IWaitable
- condition variable : IWaitable
- critical section : IWaitable (aka reentrant mutex)
- event : IWaitable
- mutex : IWaitable
- semaphore : IWaitable
- rwlock (aka shared mutex)
   IWaitable ::GetRead(),
   IWaitable ::GetWrite() 
- spinlocks


Problem one (1): <br>
    Most STL implementations have generally awful to unnecessarily inefficient abstraction. <br> <br>
    Defer to libc++'s abuse of spin while (cond) yield loops and msvc/stl's painfully slow
    std::mutex and semaphore primitives. 

Problem Two (2):  <br>
    Moving to or from linux, macos, bsd, and win32 under varous kernels, there is no one <br>
    standard (even in posix land) for the key thread primitives. 

Bonus point NT (3):  <br>
    The userland CriticalSection/CV set of APIs suck, lacking timeouts and try lock 

Bonus point UNIX (4):  <br>
    No wait multiple mechanism
    
    
1, 2, 3: Use the high performance AuThreadPrimitives objects<br>
4: Consider using loop sources, perhaps with the async subsystem, in your async application. <br>
Performance of loop sources will vary wildly between platforms, always being generally worse than <br>
the high performance primitives. They should be used to observe kernel-level signalable resources.<br>
4 ex: Windows developers could use loop sources as a replacement to WaitMultipleObjects with more overhead 
    

## Strings

The Aurora Runtime defines an `AuString` type as an `std::string`; however, it should be assumed this type<br>
represents a binary blob of UTF-8. Looking to switch to `tiny-utf8` for UTF-8 safety. 

## Memory

```
Types: 
    AuSPtr<Type_t>
    AuWPtr<Type_t>
    AuUPtr<Type_t, Deleter_t>
Functions:
    AuSPtr<T> AuMakeShared<T>(Args&& ...)
    AuSPtr<T> AuUnsafeRaiiToShared<T>(T *)
    AuSPtr<T> AuUnsafeRaiiToShared<T>(AuUPtr<T>) 
Macros:
    AuSPtr<This_T> AuSharedFromThis()
    AuWPtr<This_T> AuWeakFromThis()
    AuFunction<...> AuBindThis(This_t *::?, ...)
```

By default, AuSPtr is backed by `std::shared_ptr`, extended by `#include <Aurora/Memory/ExtendStlLikeSharedPtr>` <br>
Using this class, undefined behaviour on dereference and operator pointer is altered to guarantee an AU_THROW_STRING <br>
It would be 'nice' to live in a world without C++ exceptions; however, nothrow and attempts to mitigate them and their <br>
basis tend to result in `std::terminate` being called sooner or later. Defer to [exceptions](#exceptions) on how we log<br>
and treat them. Those who live in nothrow land can eat the exception, turning it into a terminate condition. Smarter <br>
applications may be able to catch the null dereference and continue operation without brining the whole kingdom down with it.

### Note
Aurora provides a bring your own container and shared pointer model overloadable in your configuration header.<br>
User-overloadable type declerations and generic access utilities are defined under [utilities](#utilities)<br>


## Binding

Aurora Runtime provides C++ APIs; however, it should be noted that two libraries are used to extend interfaces and enums <br>
to help with porting and internal utility access. One, AuroraEnums, wraps basic enumerations and provides value vectors; <br>
value strings; look up; iteration; and more. The other, AuroraInterfaces, provides *TWO* class types for each virtual interface. <br>
Each interface can be backed by a; C++ class method overriding a superclass's `virtual ...(...) = 0;` method, or a `AuFunctional`<br>
-based structure. <br>

It should be noted that most language bindings and generator libraries (^swig, v8pp, nbind, luabind) work with shared pointers. <br>
Other user code may wish to stuff pointers into a machineword-sized space, whether its a C library, a FFI, or a size constraint. <br>
One handle or abstraction layer will be required to integrate the C++ API into the destination platform, and assuming we have a <br>
C++ language frontend parsing our API, we can use `AuSPtr` for all caller-to-method constant reference scanerios. <br>
Furthermore, `AuSPtrs` can be created, without a deletor, using `AuUnsafeRaiiToShared(unique/raw pointer)`. To solve the raw <br>
pointer issue, `AuSPtrs` are created in the public headers with the help of exported/default visibility interface create and <br>
destroy functions. These APIs provide raw pointers to public C++ interfaces, and as such, can be binded using virtually any <br>
shim generator. Method and API mapping will likely involve manual work from the library developer to reimplement AU concepts <br>
under their language runtime instead of using the C++ platform, or at least require manual effort to shim or map each runtime <br>
prototype into something more sane across the language barrier. <br>

Memory is generally viewed through a `std::span` like concept called MemoryViews. `MemoryViewRead` and `MemoryViewWrite` <br>
provide windows into a defined address range. `MemoryViewStreamRead` and `MemoryViewStreamWrite` expand upon this concept by <br>
accepting an additional offset (`AuUInt &: reference`) that is used by internal APIs to indicate how many bytes were written or <br>
read from a given input region. Such requirement came about from so many APIs, networking, compression, encoding, doing the exact <br>
same thing in different not-so-portable ways. Unifying memory access to 4 class types should aid with SWIG prototyping. 

Unrelated note, structure interfacing with questionable C++ ABI reimplementations is somewhat sketchy in FFI projects (^ CppSharp)<br>
can lead to some memory leaks. <br>


## IO

[TODO] Summary

A note about encoding; stdin, file encoding, text decoders, and other IO resources work with<br>
codepage UTF-8 as the internal encoding scheme. String overloads and dedicated string APIs in<br>
the IO subsystem will always write BOM prefixed UTF-8 and attempt to read a BOM to translate<br>
any other input to UTF-8.

## NIO

The networking stack supports a handful of architectural paradigms<br>
- block on write<br>
- delegate write to end of network frame on write<br>
- read with an all-or-nothing flag and an async flag<br>
- read with an asynchronous stream callback
- peaking<br>
- async read/write pump whenever and/or all 

## FIO

[TODO] async, fio abstraction, utf8 read/write, blob read/write, stat, dir recursion, stream abstraction

### Paths

We assume all paths are messy. Incorrect splitters, double splitters, relative paths, and<br>
keywords are resolved internally. No such URL or path builder, data structure to hold a <br>
tokenized representation, or similar concept exists in the codebase. All string 'paths' are<br>
simply expanded, similar to MSCRT 'fullpath'/UNIX 'realpath', at time of usage. <br>

Path tokens include:<br>
[0] == '.' = cwd<br>
[0] == '~' = platform specific user directory / brand / Profile<br>
[0] == '!' = platform specific app config directory / brand / System<br>
[0] == '?' = ., !, or ~<br>
.. = go back<br>
/ = splitter<br>
\ = splitter<br>

<br>
[TODO] Aurora Branding <br>
[TODO] Aurora IO Resources <br>

## Aurora Async

The Aurora Runtime offers an optional asynchronous task driven model under the AuAsync <br>
namespace. Featuring promises, thread group pooling, functional-to-task wrapping, and <br>
task-completion callback-task-dispatch idioms built around 3 concepts.  <br>

Example:

## Proccesses

The Aurora Runtime provides worker process monitoring, worker Stdin/out stream redirection,<br>
process spawning, file opening, and url opening functionality.

## Locale

Encoding and decoding UTF-8, UTF-16, UTF-32, GBK, GB-2312, and SJIS support using platform<br>
specific APIs. Fetch system language and country backed by environment variables, the OS<br>
system configuration, the unix locale env variable, and/or the provided overload mechanism.

## Philosophies

- Assume C++17 language support in the language driver

- Use AuXXX type bindings for std types, allow customers to overload the std namespace
   We assume *some* containers and utility APIs exist, but where they come from is up to you

- Keep the code and build chain simple such that any C++ developer could maintain 
  their own software stack built around aurora components.

- Dependencies and concepts should be cross-platform, cross-architecture, cross-ring friendly
  
  It is recommended to fork and replace any legacy OS specific code with equivalent
   AuroraRuntime concepts, introducing a circular dependency with the Aurora Runtime
  
  APIs shouldn't be designed around userland, mobile computing, or desktop computing;
   AuroraRuntime must provide a common backbone for all applications. 
   
  Locale and user-info APIs will be limited due to the assumption userland is not a 
   concept

- Dependencies, excluding core reference algorithms (eg compression), must be rewritten
   and phased out over time.

- Dependencies should not be added if most platforms provide some degree of native support<br>
  Examples:<br>
            -> Don't depend on a pthread shim for windows; implement the best thread <br>
               primitives that lie on the best possible api for them <br>
            -> Don't depend on ICU when POSIX's iconv and Win32's multibyte apis cover<br>
               everything a conservative developer cares about; chinese, utf-16, utf-8,<br>
               utf-32 conversion, on top of all the ancient windows codepages

- Dependencies should only be added conservatively when it saves development time and 
  provides production hardening <br>
  Examples:<br>
            -> Use embedded crypto libraries; libtomcrypt, libtommath<br>
                -> While there are some bugs in libtomcrypt and others, none appear to <br>
                   cryptographically cripple the library. Could you do better?<br>
            -> Use portable libraries like mbedtls, O(1) heap, mimalloc<br>
                -> Writing a [D]TLS/allocator stack would take too much time<br>
                -> Linking against external allocators, small cross-platform utilities, and <br>
                   so on is probably fine <br>
            -> Shim libcurl instead of inventing yet another http stack <br>