AuroraRuntime/readme.md

## IN DEVELOPMENT

## AuroraRuntime

![picture](https://gitea.reece.sx/AuroraSupport/AuroraRuntime/raw/branch/master/Media/Hello%20Aurora.png)

The Aurora Runtime is an platform abstraction layer for cross-platform C++ development targeting
embedded and PC systems. Simply fetch a binary package for your toolchain or integrate the build
scripts into your applications build pipeline to get started. 

View this file without markdown for improved formatting

## Features

- Lightweight threading and synchronization primitives
- Async threading primitives, including WaitMultipleObjects paradigm
- Asynchronous and synchronous IO abstraction 
- Optional event driven async programming paradigm
- Console; graphical and standard; binary and UTF-8 logger
- Debug and Telementry; asserts, exception logging, fio, nio backends
- Crypto ECC/[25519, P-384, P-256], [AES, RSA, X509], [common digests]
- IPC [WIP]
- Network
- Random
- Hardware Info
- FIO settings registry
- Compression
- Locale and encoding
- C++ utility templates and macros
- Follows all strings are UTF-8 convention

## Links

API: <br>
Doxygen: <br>
Examples: <br>
Tests: <br>
Cmake-stable:<br>
Build Pipeline: https://git.reece.sx/AuroraPipeline/Build

## Utilities

Aurora Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraUtils.hpp <br>
Aurora Macro Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraMacros.hpp <br>
Aurora Overloadable Type Declerations: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraTypedefs.hpp


## Logging

~~Aurora Runtime does not attempt to implement your favourite production logger. We instead <br>
implement a subscription based log message dispatcher with some default backends including <br>
a file logger, Windows debug logging, Windows conhost stdin/out using UTF-8, UNIX stdin/out <br>
respecting the applications codepage, a wxWidgets toolkit GUI, and hopefully more to come~~<br>


Additionally, consoles that provide an input stream can be used in conjunction with the parse
subsystem to provide basic command-based deserialization, tokenization, and dispatch of UTF-8 
translated strings regardless of the system locale

## Exceptions

Through the use of compiler internal overloads, ELF hooking, and Win32 
`AddVectoredExceptionHandler`, Aurora Runtime hooks exceptions at the time of throw, including 
*some* out of ecosystem exceptions, providing detailed telemetry of the object type, object 
string, and backtrace. In addition, the `AuDebug` namespace provides TLS based last-error and 
last-backtrace methods. <br>

EXCEPTIONS ARE NOT CONTROL FLOW...<br>
- Aurora Runtime WILL attempt to mitigate exceptions in internal logic
- Aurora Runtime WILL NOT abuse exceptions to communicate failure
- Aurora Runtime WILL try to decouple internal exceptions from the API
- Aurora Runtime WILL NOT use anything that automatically crashes on exception catch (no-nothrow)
- Aurora Runtime WILL provide extended exception information to telemetry backends and through 
   the `AuDebug` namespace
- Aurora Runtime WILL NOT make any guarantees of being globally-nothrow; however, it should be a 
   safe assumption in non-critical environments 

`SysPanic` can be used to format a `std::terminate`-like exit condition, complete with telemetry 
data and safe cleanup.


## Loop

Aurora Runtime offers a main loop that connects multiple input sources into one delegate.  <br>
Timers, semaphores, mutexes, events, X11, FDs, Win32 msg loop, macos, IPC, file aio handles, and<br>
async runner main loop sources will be supported. This equates to a cross-platfom equivalent of<br>
NT's MsgWaitForMultipleObjects in the form of a MainLoop object and a WaitMultiple function.

## Thread Primitives

The Aurora Runtime provides platform optimized threading primitives inheriting from a featureful<br>
IWaitable interface. Each method is guaranteed.


```
IWaitable
   bool TryLock()
   void Lock(relativeTimeoutInMilliseconds)
   void Lock()
   void Unlock() 
```

Included high performance primitives
- arbitrary IWaitable condition variable
- condition mutex : IWaitable
- condition variable : IWaitable
- critical section : IWaitable (aka reentrant mutex)
- event : IWaitable
- mutex : IWaitable
- semaphore : IWaitable
- rwlock (aka shared mutex)
   IWaitable ::GetRead(),
   IWaitable ::GetWrite() 
- spinlocks


Problem one (1): <br>
    Most STL implementations have generally awful to unnecessarily inefficient abstraction. <br> <br>
    Defer to libc++'s abuse of spin while (cond) yield loops and msvc/stl's painfully slow
    std::mutex and semaphore primitives. 

Problem Two (2):  <br>
    Moving to or from linux, macos, bsd, and win32 under varous kernels, there is no one <br>
    standard (even in posix land) for the key thread primitives. 

Bonus point NT (3):  <br>
    The userland CriticalSection/CV set of APIs suck, lacking timeouts and try lock 

Bonus point UNIX (4):  <br>
    No wait multiple mechanism
    
    
1, 2, 3: Use the high performance AuThreadPrimitives objects<br>
4: Consider using loop sources, perhaps with the async subsystem, in your async application. <br>
Performance of loop sources will vary wildly between platforms, always being generally worse than <br>
the high performance primitives. They should be used to observe kernel-level signalable resources.<br>
4 ex: Windows developers could use loop sources as a replacement to WaitMultipleObjects with more overhead 
    

## Strings

The Aurora Runtime defines an `AuString` type as an `std::string`; however, it should be assumed this type<br>
represents a binary blob of UTF-8. Looking to switch to `tiny-utf8` for UTF-8 safety. 

## Memory

```
Types: 
    AuSPtr<Type_t>
    AuWPtr<Type_t>
    AuUPtr<Type_t, Deleter_t>
Functions:
    AuSPtr<T> AuMakeShared<T>(Args&& ...)
    AuSPtr<T> AuUnsafeRaiiToShared<T>(T *)
    AuSPtr<T> AuUnsafeRaiiToShared<T>(AuUPtr<T>) 
Macros:
    AuSPtr<This_T> AuSharedFromThis()
    AuWPtr<This_T> AuWeakFromThis()
    AuFunction<...> AuBindThis(This_t *::?, ...)
```

By default, AuSPtr is backed by `std::shared_ptr`, extended by `#include <Aurora/Memory/ExtendStlLikeSharedPtr>` <br>
Using this class, undefined behaviour on dereference and operator pointer is altered to guarantee an AU_THROW_STRING <br>
It would be 'nice' to live in a world without C++ exceptions; however, nothrow and attempts to mitigate them and their <br>
basis tend to result in `std::terminate` being called sooner or later. Defer to [exceptions](#exceptions) on how we log<br>
and treat them. Those who live in nothrow land can eat the exception, turning it into a terminate condition. Smarter <br>
applications may be able to catch the null dereference and continue operation without brining the whole kingdom down with it.

### Note
Aurora provides a bring your own container and shared pointer model overloadable in your configuration header.<br>
User-overloadable type declerations and generic access utilities are defined under [utilities](#utilities)<br>


## Binding

Aurora Runtime provides C++ APIs; however, it should be noted that two libraries are used to extend interfaces and enums
to help with porting and internal utility access. One, AuroraEnums, wraps basic enumerations and provides value vectors;
value strings; look up; iteration; and more. The other, AuroraInterfaces, provides *TWO* class types for each virtual interface.
Each interface can be backed by a; C++ class method overriding a superclass's `virtual ...(...) = 0;` method, or a `AuFunctional`
-based structure.

It should be noted that most language bindings and generator libraries (^swig, v8pp, nbind, luabind) work with shared pointers.
Other user code may wish to stuff pointers into a machineword-sized space, whether its a C library, a FFI, or a size constraint.
One handle or abstraction layer will be required to integrate the C++ API into the destination platform, and assuming we have a
C++ language frontend parsing our API, we can use `AuSPtr` for all caller-to-method constant reference scanerios.
Furthermore, `AuSPtrs` can be created, without a deletor, using `AuUnsafeRaiiToShared(unique/raw pointer)`. To solve the raw
pointer issue, `AuSPtrs` are created in the public headers with the help of exported/default visibility interface create and
destroy functions. These APIs provide raw pointers to public C++ interfaces, and as such, can be binded using virtually any
shim generator. Method and API mapping will likely involve manual work from the library developer to reimplement AU concepts
under their language runtime instead of using the C++ platform, or at least require manual effort to shim or map each runtime
prototype into something more sane across the language barrier.

Memory is generally viewed through a `std::span` like concept called MemoryViews. `MemoryViewRead` and `MemoryViewWrite`
provide windows into a defined address range. `MemoryViewStreamRead` and `MemoryViewStreamWrite` expand upon this concept by
accepting an additional offset (`AuUInt &: reference`) that is used by internal APIs to indicate how many bytes were written or
read from a given input region. Such requirement came about from so many APIs, networking, compression, encoding, doing the exact
same thing in different not-so-portable ways. Unifying memory access to 4 class types should aid with SWIG prototyping. 

Unrelated note, structure interfacing with questionable C++ ABI reimplementations is somewhat sketchy in FFI projects (^ CppSharp)
can lead to some memory leaks.


## IO

[TODO] Summary

A note about encoding; stdin, file encoding, text decoders, and other IO resources work with<br>
codepage UTF-8 as the internal encoding scheme. String overloads and dedicated string APIs in<br>
the IO subsystem will always write BOM prefixed UTF-8 and attempt to read a BOM to translate<br>
any other input to UTF-8.

## NIO

The networking stack supports a handful of architectural paradigms<br>
- block on write<br>
- delegate write to end of network frame on write<br>
- read with an all-or-nothing flag and an async flag<br>
- read with an asynchronous stream callback
- peaking<br>
- async read/write pump whenever and/or all 

## FIO

[TODO] async, fio abstraction, utf8 read/write, blob read/write, stat, dir recursion, stream abstraction

### Paths

We assume all paths are messy. Incorrect splitters, double splitters, relative paths, and<br>
keywords are resolved internally. No such URL or path builder, data structure to hold a <br>
tokenized representation, or similar concept exists in the codebase. All string 'paths' are<br>
simply expanded, similar to MSCRT 'fullpath'/UNIX 'realpath', at time of usage. <br>

Path tokens include:<br>
[0] == '.' = cwd<br>
[0] == '~' = platform specific user directory / brand / Profile<br>
[0] == '!' = platform specific app config directory / brand / System<br>
[0] == '?' = ., !, or ~<br>
.. = go back<br>
/ = splitter<br>
\ = splitter<br>

<br>
[TODO] Aurora Branding <br>
[TODO] Aurora IO Resources <br>

## Aurora Async

The Aurora Runtime offers an optional asynchronous task driven model under the AuAsync <br>
namespace. Featuring promises, thread group pooling, functional-to-task wrapping, and <br>
task-completion callback-task-dispatch idioms built around 3 concepts.  <br>

Example:

## Proccesses

The Aurora Runtime provides worker process monitoring, worker Stdin/out stream redirection,<br>
process spawning, file opening, and url opening functionality.

## Locale

Encoding and decoding UTF-8, UTF-16, UTF-32, GBK, GB-2312, and SJIS support using platform<br>
specific APIs. Fetch system language and country backed by environment variables, the OS<br>
system configuration, the unix locale env variable, and/or the provided overload mechanism.

## Philosophies

- Assume C++17 language support in the language driver

- Use AuXXX type bindings for std types, allow customers to overload the std namespace
   We assume *some* containers and utility APIs exist, but where they come from is up to you

- Keep the code and build chain simple such that any C++ developer could maintain 
  their own software stack built around aurora components.

- Dependencies and concepts should be cross-platform, cross-architecture, cross-ring friendly
  
  It is recommended to fork and replace any legacy OS specific code with equivalent
   AuroraRuntime concepts, introducing a circular dependency with the Aurora Runtime
  
  APIs shouldn't be designed around userland, mobile computing, or desktop computing;
   AuroraRuntime must provide a common backbone for all applications. 
   
  Locale and user-info APIs will be limited due to the assumption userland is not a 
   concept

- Dependencies, excluding core reference algorithms (eg compression), must be rewritten
   and phased out over time.

- Dependencies should not be added if most platforms provide some degree of native support<br>
  Examples:<br>
            -> Don't depend on a pthread shim for windows; implement the best thread <br>
               primitives that lie on the best possible api for them <br>
            -> Don't depend on ICU when POSIX's iconv and Win32's multibyte apis cover<br>
               everything a conservative developer cares about; chinese, utf-16, utf-8,<br>
               utf-32 conversion, on top of all the ancient windows codepages

- Dependencies should only be added conservatively when it saves development time and 
  provides production hardening <br>
  Examples:<br>
            -> Use embedded crypto libraries; libtomcrypt, libtommath<br>
                -> While there are some bugs in libtomcrypt and others, none appear to <br>
                   cryptographically cripple the library. Could you do better?<br>
            -> Use portable libraries like mbedtls, O(1) heap, mimalloc<br>
                -> Writing a [D]TLS/allocator stack would take too much time<br>
                -> Linking against external allocators, small cross-platform utilities, and <br>
                   so on is probably fine <br>
            -> Shim libcurl instead of inventing yet another http stack <br>