[*] Update README.md
This commit is contained in:
parent
4675be4d78
commit
d02ce62d92
422
README.md
422
README.md
@ -16,7 +16,7 @@ pipeline to get started.
|
||||
- Reduced C++ standard template library dependence (^1)
|
||||
- High performance threading and synchronization primitives (os userland sched optimized)
|
||||
- Async even driven subsystem with high perf sync primitives
|
||||
- Abstract kernel file/net transaction, IPC, timer, semaphore, et al abstraction in the form of LoopQueues (like, MacOS Run Loops)
|
||||
- Abstract kernel file/net transaction, IPC, timer, semaphore, et al abstraction in the form of LoopQueues (eg, MacOS RunLoops)
|
||||
- Asynchronous and synchronous IO (network, character, file, buffered, process, and io watcher)
|
||||
- Optional event driven async programming paradigm
|
||||
- Consoles; graphical and standard, file archives
|
||||
@ -85,19 +85,19 @@ Aurora Overloadable Type Declerations: [Main Header](https://gitea.reece.sx/Auro
|
||||
## Logging
|
||||
|
||||
Logging is implemented through 2 subsystems, console and logging. Console provides IO abstraction
|
||||
to logger subsystem sinks. Sinks are user implementable interfaces that can be either synchronous
|
||||
to the logger subsystem sinks. Sinks are user implementable interfaces that can be either synchronous
|
||||
or asynchronous. Loggers are defined as an internal object the takes logger messages and dumps
|
||||
them to the relevant subscriber.
|
||||
|
||||
Flushing occurs at a fixed rate on a low prio secondary thread without any configuration requirement.
|
||||
The resources spent on the thread is shared with the telemetry and debug subsystems to reduce
|
||||
the thread overhead of the runtime. Flushes also occur during panic events and other relevant
|
||||
problematic points.
|
||||
Flushing occurs at a fixed rate on a low prio background thread without any configuration requirement.
|
||||
The resources spent on the background thread is shared with the telemetry and debug subsystems to
|
||||
reduce the overall thread count of the runtime. Flushes also occur during panic events and other
|
||||
relevant problematic points.
|
||||
|
||||
Asynchronous logger sinks may double buffer log lines between the asynchronous callback and
|
||||
on flush, where the latter call is guaranteed after one or more delegated dispatch. The Windows
|
||||
Event Log backend takes advantage of this to multi-line group messages by approx time and log
|
||||
level.
|
||||
OnFlush callback, where the latter call is guaranteed after one or more delegated dispatch. The
|
||||
Windows Event Log backend takes advantage of this to multi-line group messages by approx time
|
||||
and log level.
|
||||
|
||||
Additionally, in the console subsystem, consoles that provide an input stream can be used in
|
||||
conjunction with the parse subsystem to provide basic command-based deserialization, tokenization,
|
||||
@ -127,6 +127,214 @@ EXCEPTIONS ARE NOT CONTROL FLOW...<br>
|
||||
`SysPanic` can be used to format a `std::terminate`-like exit condition, complete with telemetry
|
||||
data and safe cleanup.
|
||||
|
||||
## Thread Primitives
|
||||
|
||||
The Aurora Runtime provides platform optimized threading primitives inheriting from a featureful
|
||||
IWaitable interface. Each method is guaranteed. Each primitive is user-land scheduler optimized.
|
||||
|
||||
```
|
||||
struct IWaitable
|
||||
{
|
||||
virtual bool TryLock() = 0;
|
||||
virtual void Lock(relativeTimeoutInMilliseconds) = 0;
|
||||
virtual void Lock() = 0;
|
||||
virtual void Unlock() = 0;
|
||||
}
|
||||
```
|
||||
|
||||
Included high performance primitives
|
||||
- arbitrary condition variable ^1
|
||||
- condition mutex
|
||||
- condition variable
|
||||
- critical section ^2
|
||||
- event
|
||||
- mutex
|
||||
- semaphore
|
||||
- rwlock ^3
|
||||
- spinlocks
|
||||
|
||||
^1 Accepts __any__ IWaitable as the mutex \
|
||||
^2 Reentrant Mutex \
|
||||
^3 Includes extended read to write upgrades and permits write-entrant read-routines to prevent
|
||||
writer deadlocks.
|
||||
|
||||
### Fixing problems in other scheduler apis
|
||||
|
||||
Problem one (1): \
|
||||
Most STL implementations have generally awful to unnecessarily inefficient abstraction.
|
||||
Defer to libc++'s abuse of spin while (cond) yield loops and msvc/stl's painfully slow
|
||||
std::mutex and semaphore primitives.
|
||||
|
||||
Problem Two (2): \
|
||||
Moving to or from linux, macos, bsd, and win32 under varous kernels, there is no one
|
||||
standard (even in posix land) for the key thread primitives.
|
||||
|
||||
Bonus point NT (3): \
|
||||
The userland CriticalSection/CV set of APIs suck, lacking timeouts and try lock
|
||||
|
||||
Bonus point UNIX (4): \
|
||||
No wait multiple mechanism
|
||||
|
||||
|
||||
1, 2, 3: Use the high performance AuThreadPrimitives objects
|
||||
|
||||
4: Consider using loop sources, perhaps with the async subsystem, in your async application.
|
||||
Performance of loop sources will vary wildly between platforms, always being generally worse
|
||||
than the high performance primitives. They should be used to observe kernel-level signalable
|
||||
resources.
|
||||
|
||||
4 ex: Windows developers could use loop sources as a replacement to WaitMultipleObjects with
|
||||
more overhead
|
||||
|
||||
## IO
|
||||
|
||||
The Aurora Runtime implements a multiple io wait loop sub-subsystem, file io, network io,
|
||||
various adapters and connectors, io processors, io/character, io/buffered, and other such
|
||||
concepts to aid with writing low-level cross-platform IO.
|
||||
|
||||
An important note about texting encoding. Stdin, file encoding, text decoders, and other IO
|
||||
resources work with codepage UTF-8 as the internal encoding scheme. String overloads and
|
||||
dedicated string APIs in the IO subsystem will always write BOM prefixed UTF-8, and attempt
|
||||
to read a BOM to translate any other arbitrary user generated text input to UTF-8.
|
||||
|
||||
### Loop
|
||||
|
||||
The Aurora Runtime implements a kernel-scheduler optimized IO subsystem for managing GUIs,
|
||||
Network AIO, File AIO, IPC AIO, and thread synchronization objects through loop the loop subsystem.
|
||||
|
||||
ILoopSource is an interface defined by the loop subsystem for IO objects with a signalable state.
|
||||
Attached to an ILoopQueue, the ILoopQueue will provide wait-on and similar functionality; and
|
||||
subscription notifications of signal state change. ILoopQueue's are thread-safe allowing for
|
||||
cross-thread or mid-wait work scheduling. Subscription notifications allow for optimized loop
|
||||
source removal or no-action/non-removal replies from subscription implementer. In addition to
|
||||
the synchronization provided by the ILoopQueue, the ILoopSource interface permits arbitrary
|
||||
is-signaled-and-latch (TryLock) queries and timed-wait (WaitOn) calls on a per IO object basis.
|
||||
|
||||
### IPC
|
||||
|
||||
Included in the IPC subsystem are pipes, as used by AuProcesses; events; mutexes; semaphores;
|
||||
and shared memory views. IPC objects are exported by an internally generated non-standard string
|
||||
which contains platform specific information to import such object in a compatible application.
|
||||
Aurora IPC is not bound by processes bound by a common worker, instead, UNIX sockets and procfs
|
||||
are used to implement IPC within the applications namespace/sandbox.
|
||||
|
||||
### FIO
|
||||
|
||||
A simple file stream interface is provided by an Open function which accepts an Aurora path and
|
||||
an advisory lock level. However, all such functions are blocking in face of platform specific
|
||||
asynchronous alternatives. An alternative `IAsyncFileStream` is provided to supply the user with
|
||||
an supplier of `IAsyncTransaction`'s - an overlapped IO style interface for starting a read/write
|
||||
transaction, registering an APC-like callback, requesting a loop subsystem waitable object, and
|
||||
clearing the request. AIO is backed by `io_submit` under Linux, POSIX AIO under BSD, and Overlapped
|
||||
IO under NT. A glibc approach of spamming threads akin to libuv and skipping the synchronization
|
||||
on completion step isn't our style. Instead, you are reliant on the native async capabilities of the
|
||||
underlying operating system. Special consideration must be made for alignment, cached/uncached access,
|
||||
and supported file systems.
|
||||
|
||||
Additional utility functions exist outside of the two file interfaces for: stat, directory iteration,
|
||||
UTF-8 string reading and writing, blocking binary read/writes, and more.
|
||||
|
||||
### Paths
|
||||
|
||||
We assume all paths are messy. Incorrect splitters, double splitters, relative paths, and keywords are resolved internally.
|
||||
No URL or path builder, data structure to hold a tokenized URI expression, or similar concept exists in the codebase.
|
||||
All string paths are simply expanded, similar to MSCRT's `fullpath` or UNIX's `realpath`, at time of usage.
|
||||
|
||||
|
||||
| Expression | Meaning |
|
||||
|------------------|-------------------------------------|
|
||||
| `Path[0] == '.'` | Current Working Directory |
|
||||
| `Path[0] == '^'` | Executable module's Directory |
|
||||
| `Path[0] == '~'` | User Profile Storage + SDK brand |
|
||||
| `Path[0] == '!'` | All User Shared Storage + SDK brand |
|
||||
| `..` | Go up a directory |
|
||||
| `/` | Agnostic Directory Splitter |
|
||||
| `\` | Agnostic Directory Splitter |
|
||||
| `.` [SPLITTER] | Nothing |
|
||||
|
||||
|
||||
### Resources
|
||||
|
||||
The Aurora Runtime provides reports system, application, and user specific paths under the `Aurora::IO::FS` subsystem.
|
||||
These include the users home directory, a per vendor sandboxed application user directory, a per vendor sandboxed application
|
||||
all users directory, the user-installable program directory, the user's real home directory, and other such relevant paths.
|
||||
|
||||
### Networking
|
||||
|
||||
|
||||
### Character IO
|
||||
|
||||
|
||||
## Proccesses
|
||||
|
||||
The Aurora Runtime provides child process monitoring, asynchronous child stdin/out/err transactions, child synchronization
|
||||
(via a primitive threading event and an io event), process spawning, file opening, and url opening functionality.
|
||||
|
||||
## Locale
|
||||
|
||||
Encoding and decoding of UTF-8, UTF-16, UTF-32, GBK, GB-2312, and SJIS is supported through platform provided decoders.
|
||||
System localization information, including system codepage, country, and system language, is provided by the envrionment variables which
|
||||
are available, OS specific interfaces, or the user overload mechanism.
|
||||
|
||||
## Memory
|
||||
|
||||
### Allocator
|
||||
|
||||
Objects are allocated across API/Module boundaries. So long as the high level API design isn't horribly inefficient to an extent
|
||||
that cache invalidation and indirect lookups are minimalized, object-heavy code can optimized. On modern hardware, legitmate indirect
|
||||
branching versus short jumps aren't so expensive between modules in real world usage; and in combination with a fast enough allocator,
|
||||
there is little reason why couldn't achieve reasonable OOP performance through a C-with-classes-like API.
|
||||
|
||||
As for allocation, we generally expect a dependence on Microsoft's mimalloc. Linking against the Aurora Runtime in the Aurora
|
||||
Ecosystem will automatically replace global allocators with `Aurora::Memory`, which in turn, proxies any other suitable allocator
|
||||
interface with extended zero, array, and alignment respecting APIs. A suitably fast allocator, such as mimalloc, should reduce the
|
||||
cost of the OOP design.
|
||||
|
||||
### Memory Heap
|
||||
|
||||
Aurora provides a heap allocator for dividing up a large preallocated region of memory
|
||||
|
||||
### Shared Pointers
|
||||
|
||||
Memory objects, including shared pointers, and the object allocation model is defined by AuROXTL. The `AuSPtr` class template
|
||||
is backed by the standard `std::shared_ptr`, extended by `#include <auROXTL/auMemoryModel.hpp>,` in the default configuration.
|
||||
|
||||
AuROXTL memory primitives, and most STL containers, are source compatible with the base STL classes, such that any Aurora
|
||||
specific behaviour is lost during type reduction.
|
||||
|
||||
There are benefits of using the Aurora extended classes, include redefining null dereference on shared ptrs to throw
|
||||
an AU_THROW_STRING. Without support for native behaviour within the C++ driver, such features are rather expensive using the performace
|
||||
hacks we have available (outside of ripping the compiler apart to emit special debug info for hacky trap handlers). Arguably, it's an
|
||||
experiment worth trying now that modern hardware can make up for software and microcode flaws; and architecture translation. Most users
|
||||
probably wont even notice the performance loss, until it saves them from a hard crash and they realize dereferences are bloated.
|
||||
This is default behaviour, and can be easily disabled or configured from within your ecosystem's AuroraConfiguration.h to globally
|
||||
modify behaviour and subsequent ABI of the AuSPtr's.
|
||||
|
||||
|
||||
```
|
||||
Types:
|
||||
AuSPtr<Type_t>
|
||||
AuWPtr<Type_t>
|
||||
AuUPtr<Type_t, Deleter_t>
|
||||
|
||||
Functions:
|
||||
AuSPtr<T> AuMakeShared<T>(Args&& ...)
|
||||
AuSPtr<T> AuUnsafeRaiiToShared<T>(T *)
|
||||
AuSPtr<T> AuUnsafeRaiiToShared<T>(AuUPtr<T>)
|
||||
|
||||
Macros:
|
||||
_new (pseudo no-throw new operator)
|
||||
AuSPtr<This_T> AuSharedFromThis()
|
||||
AuWPtr<This_T> AuWeakFromThis()
|
||||
AuFunction<...> AuBindThis(This_t *::?, ...)
|
||||
```
|
||||
|
||||
|
||||
### Note
|
||||
Aurora provides a bring your own container and shared pointer model overloadable in your configuration header.<br>
|
||||
User-overloadable type declerations and generic access utilities are defined under [auROXTL](https://git.reece.sx/AuroraSupport/auROXTL)
|
||||
|
||||
|
||||
## Debug
|
||||
|
||||
### Error Markers
|
||||
@ -196,140 +404,6 @@ POSIX:
|
||||
|
||||
```
|
||||
|
||||
|
||||
## Loop
|
||||
|
||||
The Aurora Runtime implements a kernel-scheduler optimized IO subsystem for managing GUIs, Network AIO, File AIO, IPC AIO, and thread
|
||||
synchronization objects through loop the loop subsystem.
|
||||
|
||||
Loop sources are an interface defined by the loop subsystem for objects with a signalable state. Attached to a LoopQueue, the
|
||||
LoopQueue will provide a Wait-on related functions and subscription notifications of signal state change - allowing for optimized
|
||||
loop source removal or no-action replies from the subscription interface. In addition, the ILoopSource interface permits arbitrary
|
||||
is-signaled-and-latch (TryLock) queries and timed-wait (WaitOn) calls removing the need for loop queues during simple operations.
|
||||
|
||||
Advanced use cases of loop include the addition and removal of watched signalable objects on a remote worker thread, high performance
|
||||
async synchronization to lower performance IO, and the multiplexing of primitive IO streams.
|
||||
|
||||
|
||||
|
||||
## Thread Primitives
|
||||
|
||||
The Aurora Runtime provides platform optimized threading primitives inheriting from a featureful IWaitable interface.
|
||||
Each method is guaranteed. Each primitive is user-land scheduler optimized.
|
||||
|
||||
```
|
||||
struct IWaitable
|
||||
{
|
||||
virtual bool TryLock() = 0;
|
||||
virtual void Lock(relativeTimeoutInMilliseconds) = 0;
|
||||
virtual void Lock() = 0;
|
||||
virtual void Unlock() = 0;
|
||||
}
|
||||
```
|
||||
|
||||
Included high performance primitives
|
||||
- arbitrary condition variable ^1
|
||||
- condition mutex
|
||||
- condition variable
|
||||
- critical section ^2
|
||||
- event
|
||||
- mutex
|
||||
- semaphore
|
||||
- rwlock ^3
|
||||
- spinlocks
|
||||
|
||||
^1 Accepts __any__ IWaitable as the mutex \
|
||||
^2 Reentrant Mutex \
|
||||
^3 Includes extended read to write upgrades and permits write-entrant read-routines to prevent writer deadlocks.
|
||||
|
||||
### Fixing problems in other scheduler apis
|
||||
|
||||
Problem one (1): \
|
||||
Most STL implementations have generally awful to unnecessarily inefficient abstraction.
|
||||
Defer to libc++'s abuse of spin while (cond) yield loops and msvc/stl's painfully slow
|
||||
std::mutex and semaphore primitives.
|
||||
|
||||
Problem Two (2): \
|
||||
Moving to or from linux, macos, bsd, and win32 under varous kernels, there is no one
|
||||
standard (even in posix land) for the key thread primitives.
|
||||
|
||||
Bonus point NT (3): \
|
||||
The userland CriticalSection/CV set of APIs suck, lacking timeouts and try lock
|
||||
|
||||
Bonus point UNIX (4): \
|
||||
No wait multiple mechanism
|
||||
|
||||
|
||||
1, 2, 3: Use the high performance AuThreadPrimitives objects
|
||||
|
||||
4: Consider using loop sources, perhaps with the async subsystem, in your async application. Performance of loop sources will
|
||||
vary wildly between platforms, always being generally worse than the high performance primitives. They should be used to observe
|
||||
kernel-level signalable resources.
|
||||
|
||||
4 ex: Windows developers could use loop sources as a replacement to WaitMultipleObjects with more overhead
|
||||
|
||||
## Strings
|
||||
|
||||
The Aurora Runtime defines an `AuString` type as an `std::string`; however, it should be assumed this type represents a
|
||||
binary blob of UTF-8. Looking to switch to `tiny-utf8` for UTF-8 safety.
|
||||
|
||||
## Memory
|
||||
|
||||
### Allocator
|
||||
|
||||
Objects are allocated across API/Module boundaries. So long as the high level API design isn't horribly inefficient to an extent
|
||||
that cache invalidation and indirect lookups are minimalized, object-heavy code can optimized. On modern hardware, legitmate indirect
|
||||
branching versus short jumps aren't so expensive between modules in real world usage; and in combination with a fast enough allocator,
|
||||
there is little reason why couldn't achieve reasonable OOP performance through a C-with-classes-like API.
|
||||
|
||||
As for allocation, we generally expect a dependence on Microsoft's mimalloc. Linking against the Aurora Runtime in the Aurora
|
||||
Ecosystem will automatically replace global allocators with `Aurora::Memory`, which in turn, proxies any other suitable allocator
|
||||
with extended re[zero]allocate[aligned] with uniform deallocate APIs.
|
||||
|
||||
### Memory Heap
|
||||
|
||||
Aurora provides a heap allocator for dividing up a large preallocated region of memory
|
||||
|
||||
### Shared Pointers
|
||||
|
||||
Memory objects, including shared pointers, the object allocation model, and more is defined by AuROXTL. The `AuSPtr` class
|
||||
is backed by `std::shared_ptr`, extended by `#include <auROXTL/auMemoryModel.hpp>,` in the default configuration.
|
||||
|
||||
AuROXTL memory primitives, and most STL containers, are source compatible with the base STL classes, such that any Aurora
|
||||
specific behaviour is lost during type reduction.
|
||||
|
||||
Benefits of using the Aurora extended classes include redefining null dereference and operator pointer access to throw
|
||||
an AU_THROW_STRING __in an unconfigurated Aurora ecosystem__. Without support for native behaviour from the C++ drivers,
|
||||
such features can be rather expensive using performace hacks we have available (outside of ripping the compiler apart to emit special
|
||||
debug info for hacky trap handlers). However, it's an experiment worth trying now that modern hardware can make up for software and
|
||||
microcode flaws; and architecture translation. Again, this is default behaviour, and can be easily disabled or configured from within
|
||||
your AuroraConfiguration.h file globally.
|
||||
|
||||
|
||||
```
|
||||
Types:
|
||||
AuSPtr<Type_t>
|
||||
AuWPtr<Type_t>
|
||||
AuUPtr<Type_t, Deleter_t>
|
||||
|
||||
Functions:
|
||||
AuSPtr<T> AuMakeShared<T>(Args&& ...)
|
||||
AuSPtr<T> AuUnsafeRaiiToShared<T>(T *)
|
||||
AuSPtr<T> AuUnsafeRaiiToShared<T>(AuUPtr<T>)
|
||||
|
||||
Macros:
|
||||
_new (pseudo no-throw new operator)
|
||||
AuSPtr<This_T> AuSharedFromThis()
|
||||
AuWPtr<This_T> AuWeakFromThis()
|
||||
AuFunction<...> AuBindThis(This_t *::?, ...)
|
||||
```
|
||||
|
||||
|
||||
### Note
|
||||
Aurora provides a bring your own container and shared pointer model overloadable in your configuration header.<br>
|
||||
User-overloadable type declerations and generic access utilities are defined under [auROXTL](https://git.reece.sx/AuroraSupport/auROXTL)
|
||||
|
||||
|
||||
## Binding
|
||||
|
||||
Aurora Runtime provides C++ APIs; however, it should be noted that two libraries are used to extend interfaces and enums
|
||||
@ -359,55 +433,6 @@ Unrelated note, structure interfacing with questionable C++ ABI reimplementation
|
||||
can lead to some memory leaks.
|
||||
|
||||
|
||||
## IO
|
||||
|
||||
[TODO] Summary
|
||||
|
||||
An important note about texting encoding. Stdin, file encoding, text decoders, and other IO resources work with codepage UTF-8
|
||||
as the internal encoding scheme. String overloads and dedicated string APIs in the IO subsystem will always write BOM prefixed
|
||||
UTF-8 and attempt to read a BOM to translate any other input to UTF-8.
|
||||
|
||||
### FIO
|
||||
|
||||
[TODO] async, fio abstraction, utf8 read/write, blob read/write, stat, dir recursion, stream abstraction
|
||||
|
||||
### Paths
|
||||
|
||||
We assume all paths are messy. Incorrect splitters, double splitters, relative paths, and keywords are resolved internally.
|
||||
No URL or path builder, data structure to hold a tokenized URI expression, or similar concept exists in the codebase.
|
||||
All string 'paths' are simply expanded, similar to MSCRT's `fullpath` or UNIX's `realpath`, at time of usage.
|
||||
|
||||
|
||||
| Expression | Meaning |
|
||||
|------------------|-------------------------------------|
|
||||
| `Path[0] == '.'` | Current Working Directory |
|
||||
| `Path[0] == '^'` | Executable module's Directory |
|
||||
| `Path[0] == '~'` | User Profile Storage + SDK brand |
|
||||
| `Path[0] == '!'` | All User Shared Storage + SDK brand |
|
||||
| `..` | Go up a directory |
|
||||
| `/` | Agnostic Directory Splitter |
|
||||
| `\` | Agnostic Directory Splitter |
|
||||
| `.` [SPLITTER] | Nothing |
|
||||
|
||||
|
||||
[TODO] Aurora Branding <br>
|
||||
|
||||
### Resources
|
||||
[TODO] Aurora IO Resources <br>
|
||||
|
||||
|
||||
### NIO
|
||||
|
||||
- Worker thread delegated resolve using system resolver
|
||||
- Callback with fence-id based asynchronous write abstraction
|
||||
- Loop Source support
|
||||
|
||||
### CIO
|
||||
|
||||
|
||||
### BIO
|
||||
|
||||
|
||||
## Aurora Async
|
||||
|
||||
The Aurora Runtime offers an optional asynchronous task driven model under the AuAsync namespace. Featuring promises,
|
||||
@ -416,17 +441,10 @@ concepts.
|
||||
|
||||
Example:
|
||||
|
||||
## Proccesses
|
||||
## Strings
|
||||
|
||||
The Aurora Runtime provides worker process monitoring, worker stdin/out stream redirection process spawning, file
|
||||
opening, and url opening functionality. Further support was added for asynchronous stdin/out/err redirection, and
|
||||
a process termination event, through the use of the Loop subsystem.
|
||||
|
||||
## Locale
|
||||
|
||||
Encoding and decoding UTF-8, UTF-16, UTF-32, GBK, GB-2312, and SJIS is supported through platform provided decoders.
|
||||
Fetch system language and country backed by environment variables, the OS system configuration, the unix locale env
|
||||
variable, and/or the provided overload mechanism.
|
||||
The auROXTL header only library defines an `AuString` type as an `std::string`; however, it should be assumed this type represents
|
||||
a binary blob of UTF-8. Further locale processing is delegated to `Aurora::Locale[::Encoding]`
|
||||
|
||||
## Dependencies
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user