AuroraSupport/AuroraRuntime
2
1
Fork 0
Code Issues 27 Pull Requests Releases Wiki Activity

Merge remote-tracking branch 'origin/master'

Browse Source
This commit is contained in:
Reece Wilson 2021-10-22 10:11:49 +01:00
commit a679ecb534
1 changed files with 115 additions and 114 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

229
readme.txt → readme.md
View File

@ -2,9 +2,9 @@
## AuroraRuntime ## AuroraRuntime
The Aurora Runtime is an extensive platform abstraction layer for cross-platform C++ development The Aurora Runtime is an platform abstraction layer for cross-platform C++ development targeting<br>
across embedded and PC systems. Simply fetch a binary package for your toolchain or integrate embedded and PC systems. Simply fetch a binary package for your toolchain or integrate the build<br>
our build scripts into your applications build pipeline to get started. scripts into your applications build pipeline to get started.
## Features ## Features
@ -26,45 +26,51 @@ our build scripts into your applications build pipeline to get started.
## Links ## Links
API: API: <br>
Doxygen: Doxygen: <br>
Examples: Examples: <br>
Tests: Tests: <br>
Cmake-stable: Cmake-stable:<br>
Build Pipeline: Build Pipeline:
## Utilities ## Utilities
Aurora Runtime provides macros and some template apis to make writing common C++ idioms and Aurora Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraUtils.hpp <br>
tricks easier. Aurora Macro Sugar: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraMacros.hpp <br>
Aurora Overloadable Type Declerations: https://git.reece.sx/AuroraSupport/AuroraRuntime/src/branch/master/Include/AuroraTypedefs.hpp
## Logging ## Logging
Aurora Runtime does not attempt to implement your favourite production logger. We instead Aurora Runtime does not attempt to implement your favourite production logger. We instead<br>
implement a subscription based log message dispatcher with some default backends including implement a subscription based log message dispatcher with some default backends including<br>
a file logger, Windows debug logging, Windows conhost stdin/out using UTF-8, UNIX stdin/out a file logger, Windows debug logging, Windows conhost stdin/out using UTF-8, UNIX stdin/out<br>
respecting the applications codepage, a wxWidgets toolkit GUI, and hopefully more to come. respecting the applications codepage, a wxWidgets toolkit GUI, and hopefully more to come. <br>
Additionally, consoles that provide an input stream can be used in conjunction with the parse Additionally, consoles that provide an input stream can be used in conjunction with the parse<br>
subsystem to provide basic command-based deserialization, tokenization, and dispatch. subsystem to provide basic command-based deserialization, tokenization, and dispatch.
## Loop [WIP] ## Loop [WIP]
Aurora Runtime offers a main loop that connects multiple input sources into one delegate. Aurora Runtime offers a main loop that connects multiple input sources into one delegate. <br>
Timers, semaphores, mutexes, events, X11, FDs, Win32 msg loop, macos, IPC, file aio handles, and Timers, semaphores, mutexes, events, X11, FDs, Win32 msg loop, macos, IPC, file aio handles, and<br>
async runner main loop sources will be supported. This equates to a cross-platfom equivalent of async runner main loop sources will be supported. This equates to a cross-platfom equivalent of<br>
NT's MsgWaitForMultipleObjects in the form of a MainLoop object and a WaitMultiple function. NT's MsgWaitForMultipleObjects in the form of a MainLoop object and a WaitMultiple function.
## Thread Primitives ## Thread Primitives
The Aurora Runtime provides platform optimized threading primitives inheriting from a featureful The Aurora Runtime provides platform optimized threading primitives inheriting from a featureful<br>
IWaitable interface. Each method is guaranteed. IWaitable interface. Each method is guaranteed.
- IWaitable
::TryLock()
::Lock(timeout)
::Lock()
::Unlock()
```
IWaitable
bool TryLock()
void Lock(relativeTimeoutInMilliseconds)
void Lock()
void Unlock()
```
Included high performance primitives
- arbitrary IWaitable condition variable - arbitrary IWaitable condition variable
- condition mutex : IWaitable - condition mutex : IWaitable
- condition variable : IWaitable - condition variable : IWaitable
@ -77,152 +83,147 @@ IWaitable interface. Each method is guaranteed.
IWaitable ::GetWrite() IWaitable ::GetWrite()
- spinlocks - spinlocks
We acknowledge and wish to solve the two problems cross-platform developers frequently face.
Problem one: Problem one (1): <br>
Most STL implementations have generally awful to unnecessarily inefficient abstraction. Most STL implementations have generally awful to unnecessarily inefficient abstraction. <br> <br>
Defer to libc++'s abuse of spin while (cond) yield loops and msvc/stl's painfully slow Defer to libc++'s abuse of spin while (cond) yield loops and msvc/stl's painfully slow
std::mutex and semaphore primitives. std::mutex and semaphore primitives.
Problem Two: Problem Two (2): <br>
Moving to or from linux, macos, bsd, and win32 under varous kernels, there is no one Moving to or from linux, macos, bsd, and win32 under varous kernels, there is no one <br>
standard (even in posix land) for the key thread primitives. standard (even in posix land) for the key thread primitives.
Bonus point NT: Bonus point NT (3): <br>
The userland CriticalSection/CV set of APIs suck, lacking timeouts and try lock The userland CriticalSection/CV set of APIs suck, lacking timeouts and try lock
Bonus point UNIX: Bonus point UNIX (4): <br>
No wait multiple mechanism No wait multiple mechanism
If you wish to wait on primitives in an asynchronous application, look into runloop sources 1, 2, 4: Use the high performance AuThreadPrimitives objects<br>
4: Consider using loop sources, perhaps with the async subsystem, in your async application. <br>
Performance of loop sources will vary wildly between platforms, always being generally worse than <br>
the high performance primitives. They should be used to observe kernel-level signalable resources.<br>
4 ex: Windows developers could use loop sources as a replacement to WaitMultipleObjects with more overhead
## Strings ## Strings
Currently using a typedef of `std::string`, the Aurora Runtime is looking to switch over the The Aurora Runtime defines an `AuString` type as an `std::string`; however, it should be assumed this type<br>
string type over to `tiny-utf8`'s string type. **All** strings are assumed to be UTF-8. represents a binary blob of UTF-8. Looking to switch to `tiny-utf8` for UTF-8 safety.
## Memory ## Memory
Assumes using AuSPtr, AuWWPtr = std::xxx or app wide redefinition to an alternative smart ptr User-overloadable type declerations and generic access utilities are defined under [utilities](#utilities)<br>
implementation. Macros AU_WEAK_FROM_THIS, AU_SHARED_FROM_THIS provide decltype(this) Au[W/S]Ptrs Aurora provides a bring your own container and shared pointer model overloadable in your configuration header.
when the std::enable_shared_from_this or equivalent is utilized. UnsafeRaiiToShared converts raw
pointers to ownerless shared pointers for use with shared apis. ```
Types:
AuSPtr<Type_t>
AuWPtr<Type_t>
AuUPtr<Type_t, Deleter_t>
Functions:
AuSPtr<T> AuMakeShared<T>(Args&& ...)
AuSPtr<T> AuUnsafeRaiiToShared<T>(T *)
AuSPtr<T> AuUnsafeRaiiToShared<T>(AuUPtr<T>)
Macros:
AuSPtr<This_T> AuSharedFromThis()
AuWPtr<This_T> AuWeakFromThis()
AuFunction<...> AuBindThis(This_t *::?, ...)
```
Most Aurora Runtime APIs provide generic new and release functions should you not need the overhead of reference counting or unique ptrs <br>
However, strict C codebases would need to shim to C, perhaps using AuUnsafeRaiiToShared to convert T\*s to unsafe `AuSPtr<T>`s, the namespaced C++ functions <br>
It should be noted that most language bindings and generator libraries (^swig, v8pp, nbind, luabind) work with shared pointers. <br>
## IO ## IO
The IO subsystem consists of three interfaces, StreamReaders, StreamWriters, and [TODO] Summary
ArbitraryStreamReaders; an FS namespace; a network namespace; and an optional
async file io namespace.
A note about encoding; stdin, file encoding, text decoders, and other IO resources work with A note about encoding; stdin, file encoding, text decoders, and other IO resources work with<br>
codepage UTF-8 as the internal encoding scheme. String overloads and dedicated string APIs in codepage UTF-8 as the internal encoding scheme. String overloads and dedicated string APIs in<br>
the IO subsystem will always write BOM prefixed UTF-8 and attempt to read a BOM to translate the IO subsystem will always write BOM prefixed UTF-8 and attempt to read a BOM to translate<br>
any other input to UTF-8. any other input to UTF-8.
## NIO ## NIO
The networking stack supports a handful of architectural paradigms The networking stack supports a handful of architectural paradigms<br>
- block on write - block on write<br>
- delegate write to end of network frame on write - delegate write to end of network frame on write<br>
- read with an all-or-nothing flag and an async flag - read with an all-or-nothing flag and an async flag<br>
- read with an asynchronous stream callback - read with an asynchronous stream callback
- peaking - peaking<br>
- async read/write pump whenever or all - async read/write pump whenever or all
## FIO ## FIO
We suspect most developers delegate IO to a worker thread, don't wish to deal with an extern [TODO] async, fio abstraction, utf8 read/write, blob read/write, stat, dir recursion, stream abstraction
async model nor want excessively buffered streams, therefore the FIO API implements a read/write
seekable C-like file stream interface. An alternative AFIO namespace exists for the few platforms
that provide posix AIO, win32 overlap, or linux's io syscalls.
File stream, buffered read/write utilities, stat, exists, copy, move, remove, and readdir backed
by the best platform specific blocking apis.
Guaranteed 64-bit file pointer safety across seek functions.
[Open]Write to a file under a path of missing directories guarantees creation of missing sub
directories.
### Paths ### Paths
We assume all paths are messy. Incorrect splitters, double splitters, relative paths, and We assume all paths are messy. Incorrect splitters, double splitters, relative paths, and<br>
keywords are resolved internally. No such URL or path builder, data structure to hold a keywords are resolved internally. No such URL or path builder, data structure to hold a <br>
tokenized representation, or similar concept exists in the codebase. All string 'paths' are tokenized representation, or similar concept exists in the codebase. All string 'paths' are<br>
simply expanded, similar to MSCRT 'fullpath'/UNIX 'realpath', at time of usage. simply expanded, similar to MSCRT 'fullpath'/UNIX 'realpath', at time of usage. <br>
Path tokens include: Path tokens include:<br>
[0] == '.' = cwd [0] == '.' = cwd<br>
[0] == '~' = platform specific user directory / brand / Profile [0] == '~' = platform specific user directory / brand / Profile<br>
[0] == '!' = platform specific app config directory / brand / System [0] == '!' = platform specific app config directory / brand / System<br>
[0] == '?' = ., !, or ~ [0] == '?' = ., !, or ~<br>
.. = go back .. = go back<br>
/ = splitter / = splitter<br>
\ = splitter \ = splitter
## Aurora Async ## Aurora Async
The Aurora Runtime offers an optional asynchronous task driven model under the Aurora::Async The Aurora Runtime offers an optional asynchronous task driven model under the AuAsync <br>
namespace. Featuring promises, thread group pooling, functional-to-task wrapping, and namespace. Featuring promises, thread group pooling, functional-to-task wrapping, and <br>
task-completion callback-task-dispatch idioms built around 3 concepts. task-completion callback-task-dispatch idioms built around 3 concepts. <br>
Jobs are callee provided interfaces providing ::onSuccess/onFailure(const in, const out)
Tasks are an internally-provided interface providing optional<out> onFrame(const in)
WorkItems adapt a job and a task into one concept. They take a minimum sched delay, promises,
initial delay, and other requirements; to provide a Dispatch routine capable of handling
promises and other abstract developer requirements
We will not define a standard way to use these concepts. Example:
For instance, there is no guidance on whether or not you should use public task state structs
and request a Task pointer from a public API or you should accept Job and return a WorkItem;
when or where you should dispatch; whether or not to use C or Functional interfaces.
## Proccesses ## Proccesses
The Aurora Runtime provides worker process monitoring, worker Stdin/out stream redirection, The Aurora Runtime provides worker process monitoring, worker Stdin/out stream redirection,<br>
process spawning, file opening, and url opening functionality. process spawning, file opening, and url opening functionality.
## Locale ## Locale
Encoding and decoding UTF-8, UTF-16, UTF-32, GBK, GB-2312, and SJIS support using platform Encoding and decoding UTF-8, UTF-16, UTF-32, GBK, GB-2312, and SJIS support using platform<br>
specific APIs. Fetch system language and country backed by environment variables, the OS specific APIs. Fetch system language and country backed by environment variables, the OS<br>
system configuration, the unix locale env variable, and/or the provided overload mechanism. system configuration, the unix locale env variable, and/or the provided overload mechanism.
## Philosophies ## Philosophies
- Assume C++17 language support in the language driver - Assume C++17 language support in the language driver
- Solve the large issues nobody is tackling. To avoid reinventing the wheel accept third party <br>
- To avoid reinventing the wheel, solve the large issues nobody is tackling, and accept solutions when the pros (developer time) weighted against the negatives (legal, bulk) makes
third party solutions weighted against relevant legal constraints and developer time sense.
- Use AuXXX type bindings for std types, allow customers to overload the std namespace - Use AuXXX type bindings for std types, allow customers to overload the std namespace
- Keep the code and build chain simple such that any C++ developer could maintain - Keep the code and build chain simple such that any C++ developer could maintain
their own software stack built around aurora components. their own software stack built around aurora components.
- Dependencies should be cross-platform friendly - Dependencies should be cross-platform friendly
It is recommended to fork and replace any legacy OS specific code with equivalent It is recommended to fork and replace any legacy OS specific code with equivalent
AuroraRuntime concepts AuroraRuntime concepts
- Dependencies, excluding core reference algorithms (eg compression), must be rewritten - Dependencies, excluding core reference algorithms (eg compression), must be rewritten
and phased out over time. and phased out over time.
- Dependencies should not be added if most platforms provide some degree of native support<br>
- Dependencies should not be added if most platforms provide some degree of native support Examples:<br>
Examples: -> Don't depend on a pthread shim for windows; implement the best thread -> Don't depend on a pthread shim for windows; implement the best thread <br>
primitives that lie on the best possible api for them primitives that lie on the best possible api for them <br>
Don't depend on ICU when POSIX's iconv and Win32's multibyte apis cover Don't depend on ICU when POSIX's iconv and Win32's multibyte apis cover<br>
everything a conservative developer cares about; chinese, utf-16, utf-8, everything a conservative developer cares about; chinese, utf-16, utf-8,<br>
utf-32 conversion, on top of all the ancient windows codepages utf-32 conversion, on top of all the ancient windows codepages
- Dependencies should only be added conservatively when it saves development time and - Dependencies should only be added conservatively when it saves development time and
provides production hardening provides production hardening <br>
Examples: -> Use embedded crypto libraries; libtomcrypt, libtommath Examples:<br>
-> While there are some bugs in libtomcrypt and others, none appear to -> Use embedded crypto libraries; libtomcrypt, libtommath<br>
cryptographically cripple the library. Could you do better? -> While there are some bugs in libtomcrypt and others, none appear to <br>
-> Use portable libraries like mbedtls, O(1) heap, mimalloc cryptographically cripple the library. Could you do better?<br>
-> Writing a [D]TLS/allocator stack would take too much time -> Use portable libraries like mbedtls, O(1) heap, mimalloc<br>
-> Linking against external allocators, small cross-platform utilities, and -> Writing a [D]TLS/allocator stack would take too much time<br>
so on is probably fine -> Linking against external allocators, small cross-platform utilities, and <br>
-> Shim libcurl instead of inventing yet another http stack so on is probably fine <br>
-> Shim libcurl instead of inventing yet another http stack <br>