AuroraRuntime/readme.md

230 lines
9.4 KiB
Markdown
Raw Normal View History

2021-09-06 10:58:08 +00:00
## IN DEVELOPMENT
## AuroraRuntime
2021-10-21 16:20:52 +00:00
The Aurora Runtime is an platform abstraction layer for cross-platform C++ development targeting<br>
embedded and PC systems. Simply fetch a binary package for your toolchain or integrate the build<br>
scripts into your applications build pipeline to get started.
2021-09-06 10:58:08 +00:00
## Features
- Lightweight threading primitives backed by OS specific backends
- Async threading primitives, including WaitMultipleObjects paradigm [WIP]
- Async and regular IO abstraction
- Optional event driven async programming paradigm
- Console; graphical and standard; binary and UTF-8 logger
- Debug and Telementry; asserts, exception logging, fio, nio backends
- Crypto ECC/[25519, P-384, P-256], [AES, RSA, X509], [common digests]
- IPC [WIP]
- Network
- Random
- Hardware Info
- FIO settings registry
- Compression
- Locale and encoding
- C++ utility templates and macros
## Links
2021-10-21 16:20:52 +00:00
API: <br>
Doxygen: <br>
Examples: <br>
Tests: <br>
Cmake-stable:<br>
2021-09-06 10:58:08 +00:00
Build Pipeline:
## Utilities
2021-10-21 16:20:52 +00:00
Aurora Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraUtils.hpp <br>
Aurora Macro Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraMacros.hpp <br>
Aurora Overloadable Type Declerations: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraTypedefs.hpp
2021-09-06 10:58:08 +00:00
## Logging
2021-10-21 16:20:52 +00:00
Aurora Runtime does not attempt to implement your favourite production logger. We instead<br>
implement a subscription based log message dispatcher with some default backends including<br>
a file logger, Windows debug logging, Windows conhost stdin/out using UTF-8, UNIX stdin/out<br>
respecting the applications codepage, a wxWidgets toolkit GUI, and hopefully more to come. <br>
Additionally, consoles that provide an input stream can be used in conjunction with the parse<br>
2021-09-06 10:58:08 +00:00
subsystem to provide basic command-based deserialization, tokenization, and dispatch.
## Loop [WIP]
2021-10-21 16:20:52 +00:00
Aurora Runtime offers a main loop that connects multiple input sources into one delegate. <br>
Timers, semaphores, mutexes, events, X11, FDs, Win32 msg loop, macos, IPC, file aio handles, and<br>
async runner main loop sources will be supported. This equates to a cross-platfom equivalent of<br>
2021-09-06 10:58:08 +00:00
NT's MsgWaitForMultipleObjects in the form of a MainLoop object and a WaitMultiple function.
## Thread Primitives
2021-10-21 16:20:52 +00:00
The Aurora Runtime provides platform optimized threading primitives inheriting from a featureful<br>
2021-09-06 10:58:08 +00:00
IWaitable interface. Each method is guaranteed.
2021-10-21 16:20:52 +00:00
```
IWaitable
bool TryLock()
void Lock(relativeTimeoutInMilliseconds)
void Lock()
void Unlock()
```
Included high performance primitives
2021-09-06 10:58:08 +00:00
- arbitrary IWaitable condition variable
- condition mutex : IWaitable
- condition variable : IWaitable
- critical section : IWaitable (aka reentrant mutex)
- event : IWaitable
- mutex : IWaitable
- semaphore : IWaitable
- rwlock (aka shared mutex)
IWaitable ::GetRead(),
IWaitable ::GetWrite()
- spinlocks
2021-10-21 16:20:52 +00:00
Problem one (1): <br>
Most STL implementations have generally awful to unnecessarily inefficient abstraction. <br> <br>
2021-09-06 10:58:08 +00:00
Defer to libc++'s abuse of spin while (cond) yield loops and msvc/stl's painfully slow
std::mutex and semaphore primitives.
2021-10-21 16:20:52 +00:00
Problem Two (2): <br>
Moving to or from linux, macos, bsd, and win32 under varous kernels, there is no one <br>
2021-09-18 20:10:25 +00:00
standard (even in posix land) for the key thread primitives.
2021-09-06 10:58:08 +00:00
2021-10-21 16:20:52 +00:00
Bonus point NT (3): <br>
2021-09-18 20:10:25 +00:00
The userland CriticalSection/CV set of APIs suck, lacking timeouts and try lock
2021-09-06 10:58:08 +00:00
2021-10-21 16:20:52 +00:00
Bonus point UNIX (4): <br>
2021-09-06 10:58:08 +00:00
No wait multiple mechanism
2021-10-21 16:20:52 +00:00
2021-10-23 20:13:40 +00:00
1, 2, 3: Use the high performance AuThreadPrimitives objects<br>
2021-10-21 16:20:52 +00:00
4: Consider using loop sources, perhaps with the async subsystem, in your async application. <br>
Performance of loop sources will vary wildly between platforms, always being generally worse than <br>
the high performance primitives. They should be used to observe kernel-level signalable resources.<br>
4 ex: Windows developers could use loop sources as a replacement to WaitMultipleObjects with more overhead
2021-09-06 10:58:08 +00:00
## Strings
2021-10-21 16:20:52 +00:00
The Aurora Runtime defines an `AuString` type as an `std::string`; however, it should be assumed this type<br>
represents a binary blob of UTF-8. Looking to switch to `tiny-utf8` for UTF-8 safety.
2021-09-06 10:58:08 +00:00
## Memory
2021-10-21 16:26:59 +00:00
User-overloadable type declerations and generic access utilities are defined under [utilities](#utilities)<br>
2021-10-21 16:20:52 +00:00
Aurora provides a bring your own container and shared pointer model overloadable in your configuration header.
```
Types:
AuSPtr<Type_t>
AuWPtr<Type_t>
AuUPtr<Type_t, Deleter_t>
Functions:
AuSPtr<T> AuMakeShared<T>(Args&& ...)
AuSPtr<T> AuUnsafeRaiiToShared<T>(T *)
AuSPtr<T> AuUnsafeRaiiToShared<T>(AuUPtr<T>)
Macros:
AuSPtr<This_T> AuSharedFromThis()
AuWPtr<This_T> AuWeakFromThis()
AuFunction<...> AuBindThis(This_t *::?, ...)
```
Most Aurora Runtime APIs provide generic new and release functions should you not need the overhead of reference counting or unique ptrs <br>
However, strict C codebases would need to shim to C, perhaps using AuUnsafeRaiiToShared to convert T\*s to unsafe `AuSPtr<T>`s, the namespaced C++ functions <br>
It should be noted that most language bindings and generator libraries (^swig, v8pp, nbind, luabind) work with shared pointers. <br>
2021-09-06 10:58:08 +00:00
## IO
2021-10-21 16:20:52 +00:00
[TODO] Summary
2021-09-06 10:58:08 +00:00
2021-10-21 16:20:52 +00:00
A note about encoding; stdin, file encoding, text decoders, and other IO resources work with<br>
codepage UTF-8 as the internal encoding scheme. String overloads and dedicated string APIs in<br>
the IO subsystem will always write BOM prefixed UTF-8 and attempt to read a BOM to translate<br>
2021-09-06 10:58:08 +00:00
any other input to UTF-8.
## NIO
2021-10-21 16:20:52 +00:00
The networking stack supports a handful of architectural paradigms<br>
- block on write<br>
- delegate write to end of network frame on write<br>
- read with an all-or-nothing flag and an async flag<br>
2021-09-06 10:58:08 +00:00
- read with an asynchronous stream callback
2021-10-21 16:20:52 +00:00
- peaking<br>
2021-09-06 10:58:08 +00:00
- async read/write pump whenever or all
## FIO
2021-10-21 16:20:52 +00:00
[TODO] async, fio abstraction, utf8 read/write, blob read/write, stat, dir recursion, stream abstraction
2021-09-06 10:58:08 +00:00
### Paths
2021-10-21 16:20:52 +00:00
We assume all paths are messy. Incorrect splitters, double splitters, relative paths, and<br>
keywords are resolved internally. No such URL or path builder, data structure to hold a <br>
tokenized representation, or similar concept exists in the codebase. All string 'paths' are<br>
simply expanded, similar to MSCRT 'fullpath'/UNIX 'realpath', at time of usage. <br>
Path tokens include:<br>
[0] == '.' = cwd<br>
[0] == '~' = platform specific user directory / brand / Profile<br>
[0] == '!' = platform specific app config directory / brand / System<br>
[0] == '?' = ., !, or ~<br>
.. = go back<br>
/ = splitter<br>
2021-09-06 10:58:08 +00:00
\ = splitter
## Aurora Async
2021-10-21 16:20:52 +00:00
The Aurora Runtime offers an optional asynchronous task driven model under the AuAsync <br>
namespace. Featuring promises, thread group pooling, functional-to-task wrapping, and <br>
task-completion callback-task-dispatch idioms built around 3 concepts. <br>
Example:
2021-09-06 10:58:08 +00:00
## Proccesses
2021-10-21 16:20:52 +00:00
The Aurora Runtime provides worker process monitoring, worker Stdin/out stream redirection,<br>
2021-09-06 10:58:08 +00:00
process spawning, file opening, and url opening functionality.
## Locale
2021-10-21 16:20:52 +00:00
Encoding and decoding UTF-8, UTF-16, UTF-32, GBK, GB-2312, and SJIS support using platform<br>
specific APIs. Fetch system language and country backed by environment variables, the OS<br>
2021-09-18 20:10:25 +00:00
system configuration, the unix locale env variable, and/or the provided overload mechanism.
2021-09-06 10:58:08 +00:00
## Philosophies
- Assume C++17 language support in the language driver
2021-10-21 16:20:52 +00:00
- Solve the large issues nobody is tackling. To avoid reinventing the wheel accept third party <br>
solutions when the pros (developer time) weighted against the negatives (legal, bulk) makes
sense.
2021-09-06 10:58:08 +00:00
- Use AuXXX type bindings for std types, allow customers to overload the std namespace
- Keep the code and build chain simple such that any C++ developer could maintain
their own software stack built around aurora components.
2021-09-06 10:58:08 +00:00
- Dependencies should be cross-platform friendly
It is recommended to fork and replace any legacy OS specific code with equivalent
AuroraRuntime concepts
- Dependencies, excluding core reference algorithms (eg compression), must be rewritten
and phased out over time.
2021-10-21 16:20:52 +00:00
- Dependencies should not be added if most platforms provide some degree of native support<br>
Examples:<br>
-> Don't depend on a pthread shim for windows; implement the best thread <br>
primitives that lie on the best possible api for them <br>
Don't depend on ICU when POSIX's iconv and Win32's multibyte apis cover<br>
everything a conservative developer cares about; chinese, utf-16, utf-8,<br>
2021-09-06 10:58:08 +00:00
utf-32 conversion, on top of all the ancient windows codepages
- Dependencies should only be added conservatively when it saves development time and
2021-10-21 16:20:52 +00:00
provides production hardening <br>
Examples:<br>
-> Use embedded crypto libraries; libtomcrypt, libtommath<br>
-> While there are some bugs in libtomcrypt and others, none appear to <br>
cryptographically cripple the library. Could you do better?<br>
-> Use portable libraries like mbedtls, O(1) heap, mimalloc<br>
-> Writing a [D]TLS/allocator stack would take too much time<br>
-> Linking against external allocators, small cross-platform utilities, and <br>
so on is probably fine <br>
-> Shim libcurl instead of inventing yet another http stack <br>