AuroraSupport/AuroraRuntime
2
1
Fork 0
Code Issues 27 Pull Requests Releases Wiki Activity

[*/+/-] MEGA COMMIT. ~2 weeks compressed.

Browse Source 
The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api.

[+] AuMakeSharedArray
[+] Technet ArgvQuote
[+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.)
[+] auEndianness -> Endian swap utils
[+] AuGet<N>(...)
[*] AUE_DEFINE conversion for
        ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump
[+] ConsoleMessage ByteBuffer serialization
[+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks
[*] Split logger from console subsystem
[+] StartupParameters -> A part of a clean up effort under Process
[*] Refactor SysErrors header + get caller hack
[+] Atomic APIs
[+] popcnt
[+] Ring Buffer sink
[+] Added more standard errors
        Catch,
        Submission,
        LockError,
        NoAccess,
        ResourceMissing,
        ResourceLocked,
        MalformedData,
        InSandboxContext,
        ParseError

[+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace
[+] IExitSubscriber, ETriggerLevel
[*] Write bias the high performance RWLockImpl read-lock operation operation
[+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem)
[*] Updated API style
        Digests
[+] CpuId::CpuBitCount
[+] GetUserProgramsFolder
[+] GetPackagePath
[*] Split IStreamReader with an inl file
[*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer
[+] ICharacterProvider
[+] ICharacterProviderEx
[+] IBufferedCharacterConsumer
[+] ProviderFromSharedString
[+] ProviderFromString
[+] BufferConsumerFromProvider
[*] Parse Subsystem uses character io bufferer
[*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore]
[+] ByteBuffer::ResetReadPointer
[*] Bug fix bytebuffer base not reset on free and some scaling issues
[+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section
[*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing
[+] Added 64 *byte* fast RNG seeds
[+] File Advisorys/File Lock Awareness
[+] Added extended IAuroraThread from OS identifier caches for debug purposes
[*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions.
[*] Broke AuroraUtils/Typedefs out into a separate library
[*] Update build script
[+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media
[*] Improved public API documentation
[*] Bug fix `SetConsoleCtrlHandler`
[+] Locale TimeDateToFileNameISO8601
[+] Console config stdOutShortTime
[*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl)
[*] Bug fixes in decoders
[*] Major bug fix, AuMax
[+] RateLimiter
[+] Binary file sink
[+] Log directory sink
[*] Data header usability (more operators)
[+] AuRemoveRange
[+] AuRemove
[+] AuTryRemove
[+] AuTryRemoveRange
[+] auCastUtils
[+] Finish NewLSWin32Source
[+] AuTryFindByTupleN, AuTryRemoveByTupleN
[+] Separated AuRead/Write types, now in auTypeUtils
[+] Added GetPosition/SetPosition to FileWriter
[*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp
[*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position)
[*] Hack back in the sched deinit
[+] File AIO loop source interop
[+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw
[+] Stub code for networking
[+] Compression BaseStream/IngestableStreamBase
[*] Major: read/write locks now support write-entrant read routines.
[*] Compression subsystem now uses the MemoryView concept
[*] Rewrite the base stream compressions, made them less broken
[*] Update hashing api
[*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing
[+] Added new AuByteBuffer apis
    Trim, Pad, WriteFrom, WriteString, [TODO: ReadString]
[+] Added ByteBufferPushReadState
[+] Added ByteBufferPushWriteState
[*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16.
[*] ELogLevel is now an Aurora enum
[+] Raised arbitrary limit in header to 255, the max filter buffer
[+] Explicit GZip support
[+] Explicit Zip support
[+] Added [some] compressors

et al
This commit is contained in:
Reece Wilson 2022-02-17 00:11:40 +00:00
parent 19ebdf3761
commit cf70f0d45c
410 changed files with 14066 additions and 5180 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Aurora.json
View File

@ -15,7 +15,7 @@
{
"filter": {"platforms": "win32"},
"then": {
"links": ["Bcrypt.lib", "UxTheme.lib", "Aux_ulib.lib", "Dbghelp.lib", "ws2_32.lib", "Ntdll.lib"]
"links": ["Bcrypt.lib", "UxTheme.lib", "Aux_ulib.lib", "Dbghelp.lib", "ws2_32.lib", "Ntdll.lib", "Wer.lib", "wintrust.lib"]
}
}
]

2
Include/Aurora/Async/AsyncTypes.hpp
View File

@ -9,7 +9,7 @@
namespace Aurora::Loop
{
class ILoopSource;
struct ILoopSource;
}
namespace Aurora::Async

3
Include/Aurora/Async/IThreadPool.hpp
View File

@ -51,9 +51,10 @@ namespace Aurora::Async
virtual WorkerId_t GetCurrentThread() = 0;
// Synchronization
// Note: syncing to yourself will nullify requireSignal to prevent deadlock
// Note: syncing to yourself will nullify requireSignal to prevent deadlock conditions
virtual bool Sync(WorkerId_t workerId, AuUInt32 timeoutMs = 0, bool requireSignal = false) = 0;
virtual void Signal(WorkerId_t workerId) = 0;
virtual AuSPtr<Loop::ILoopSource> WorkerToLoopSource(WorkerId_t id) = 0;
virtual void SyncAllSafe() = 0;
// Features

1
Include/Aurora/Async/IWorkItem.hpp
View File

@ -32,6 +32,7 @@ namespace Aurora::Async
// ns = time relative to the time at which the work item would otherwise dispatch
virtual AuSPtr<IWorkItem> AddDelayTimeNs(AuUInt64 ns) = 0;
// inverted WaitFor
virtual AuSPtr<IWorkItem> Then(const AuSPtr<IWorkItem> &next) = 0;
virtual AuSPtr<IWorkItem> Dispatch() = 0;

16
Include/Aurora/Async/JobFrom.hpp
View File

@ -37,7 +37,7 @@ namespace Aurora::Async
FJob<AuTuple<Args...>, Out_t> ret;
ret.onSuccess = [=](const AuTuple<Args...> &in, const Out_t &a)
{
std::apply(onSuccess, std::tuple_cat(in, AuMakeTuple<const Out_t &>(a)));
AuTupleApply(onSuccess, AuTupleCat(in, AuMakeTuple<const Out_t &>(a)));
};
return ret;
}
@ -48,12 +48,12 @@ namespace Aurora::Async
FJob<AuTuple<Args...>, Out_t> ret;
ret.onSuccess = [=](const AuTuple<Args...> &in, const Out_t &a)
{
std::apply(onSuccess, std::tuple_cat(in, AuMakeTuple<const Out_t &>(a)));
AuTupleApply(onSuccess, AuTupleCat(in, AuMakeTuple<const Out_t &>(a)));
};
ret.onFailure = [=](const AuTuple<Args...> &in)
{
std::apply(onFailure, in);
AuTupleApply(onFailure, in);
};
return ret;
}
@ -115,7 +115,7 @@ namespace Aurora::Async
FJob<AuTuple<Arg0_t, Args...>, ReturnValue_t> ret;
ret.onSuccess = [=](const AuTuple<Arg0_t, Args...> &in, const ReturnValue_t &out)
{
onSuccess(std::get<0>(in), out);
onSuccess(AuGet<0>(in), out);
};
return ret;
}
@ -126,12 +126,12 @@ namespace Aurora::Async
FJob<AuTuple<Arg0_t, Args...>, ReturnValue_t> ret;
ret.onSuccess = [=](const AuTuple<Arg0_t, Args...> &in, const ReturnValue_t &out)
{
onSuccess(std::get<0>(in), out);
onSuccess(AuGet<0>(in), out);
};
ret.onFailure = [=](const AuTuple<Arg0_t, Args...> &in)
{
onFailure(std::get<0>(in));
onFailure(AuGet<0>(in));
};
return ret;
}
@ -154,7 +154,7 @@ namespace Aurora::Async
FJob<AuTuple<AuSPtr<Clazz_t>, Args...>, ReturnValue_t> ret;
ret.onSuccess = [=](const AuTuple<AuSPtr<Clazz_t>, Args...> &in, const ReturnValue_t &out)
{
std::apply(onSuccess, std::tuple_cat(AuTuplePopFront(in), AuMakeTuple<const ReturnValue_t &>(out)));
AuTupleApply(onSuccess, AuTupleCat(AuTuplePopFront(in), AuMakeTuple<const ReturnValue_t &>(out)));
};
return ret;
}
@ -177,7 +177,7 @@ namespace Aurora::Async
FJob<AuTuple<Args...>, ReturnValue_t> ret;
ret.onSuccess = [=](const AuTuple<Args...> &in, const ReturnValue_t &out)
{
std::apply(onSuccess, std::tuple_cat(in, AuMakeTuple<const ReturnValue_t &>(out)));
AuTupleApply(onSuccess, AuTupleCat(in, AuMakeTuple<const ReturnValue_t &>(out)));
};
return ret;
}

8
Include/Aurora/Async/TaskFrom.hpp
View File

@ -34,7 +34,7 @@ namespace Aurora::Async
FTask<AuTuple<Args...>, Out_t> ret;
ret.onFrame = [callable = func](const AuTuple<Args...> &in) -> Out_t
{
return std::apply(callable, in);
return AuTupleApply(callable, in);
};
return ret;
}
@ -45,7 +45,7 @@ namespace Aurora::Async
FTask<AuTuple<Owner_t, Args...>, Out_t> ret;
ret.onFrame = [callable = func](const AuTuple<Args...> &in) -> Out_t
{
return std::apply(callable, AuTuple_cat(AuMakeTuple<Owner_t>(ownerToPin), in));
return AuTupleApply(callable, AuTupleCat(AuMakeTuple<Owner_t>(ownerToPin), in));
};
return ret;
}
@ -56,7 +56,7 @@ namespace Aurora::Async
Task_t ret;
ret.onFrame = [callable = func](const auto &in) -> ReturnValue_t
{
return std::apply(callable, AuTuplePopFront(in));
return AuTupleApply(callable, AuTuplePopFront(in));
};
return ret;
}
@ -67,7 +67,7 @@ namespace Aurora::Async
Task_t ret;
ret.onFrame = [callable = func](const auto &in) -> ReturnValue_t
{
return std::apply(callable, AuTuplePopFront(in));
return AuTupleApply(callable, AuTuplePopFront(in));
};
return ret;
}

59
Include/Aurora/CmdLine/CmdLine.hpp Normal file
View File

@ -0,0 +1,59 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: CmdLine.hpp
Date: 2022-1-31
Author: Reece
Note: Even kernels have a commandline so why not?
***/
#pragma once
namespace Aurora::CmdLine
{
/**
* @brief Returns a UTF-8 string array of argv[1, ...]
* @return
*/
AUKN_SYM const AuList<AuString> &GetCommandLineArguments();
/**
* @brief Performs a check on whether the exact key matches an argument
* @param key
* @return
*/
AUKN_SYM bool HasFlag(const AuString &key);
/**
* @brief Performs a check on whether such string came before an equals sign
* @param key
* @return
*/
AUKN_SYM bool HasValue(const AuString &key);
/**
* @brief Returns part after key= or defaultDefault
* @param key
* @param defaultValue
* @return
*/
AUKN_SYM const AuString &GetValue(const AuString &key, const AuString &defaultValue);
/**
* @brief Returns part after key= or an empty string
* @param key
* @return
*/
AUKN_SYM const AuString &GetValue(const AuString &key);
/**
* @brief Returns a constant array of flag keys
* @return
*/
AUKN_SYM const AuList<AuString> &GetFlags();
/**
* @brief Returns a constant array of value keys
* @return
*/
AUKN_SYM const AuList<AuString> &GetValues();
}

14
Include/Aurora/Compression/BasicCompression.hpp
View File

@ -12,20 +12,10 @@ namespace Aurora::Compression
/**
Compresses an in memory blob with zstandard
*/
AUKN_SYM bool Compress(const void *buffer, AuUInt32 length, Memory::ByteBuffer &out, int compressionLevel = 3);
/**
Compresses an in memory blob with zstandard
*/
AUKN_SYM bool Compress(const Memory::ByteBuffer &in, Memory::ByteBuffer &out, int compressionLevel = 3);
AUKN_SYM bool Compress(const Memory::MemoryViewRead &source, Memory::ByteBuffer &out, int compressionLevel = 3);
/**
Decompresses an in memory blob with zstandard
*/
AUKN_SYM bool Decompress(const void *buffer, AuUInt32 length, Memory::ByteBuffer &out);
/**
Decompresses an in memory blob with zstandard
*/
AUKN_SYM bool Decompress(const Memory::ByteBuffer &in, Memory::ByteBuffer &out);
AUKN_SYM bool Decompress(const Memory::MemoryViewRead &source, Memory::ByteBuffer &out);
}

48
Include/Aurora/Compression/Compression.hpp
View File

@ -7,6 +7,44 @@
***/
#pragma once
// Types
#include "ECompresionType.hpp"
#include "CompressionInfo.hpp"
// Legacy API
#include "StreamPipeProcessor.hpp"
// Recommended API
#include "ICompressionStream.hpp"
#include "StreamProcessor.hpp"
// Utility
#include "BasicCompression.hpp"
// TODO: neat comment + real world data
/*
Ballpark figures of real world performance:
https://www.gaia-gis.it/fossil/librasterlite2/wiki?name=benchmarks+(2019+update)
@ -24,13 +62,3 @@
ZSTD -> Standard use (~1.5GB/s to 2.5GB/s, respective compression ratios 2.4 and 2.1. Can be pushed to ~10 at around 750MB/s. Great general use. )
ZIP -> Zlib has a disgusting decompression upper limit of around 450MB/s for 2.7
*/
#include "BasicCompression.hpp"
#include "ECompresionType.hpp"
#include "CompressionInfo.hpp"
#include "StreamPipeProcessor.hpp"
#include "ICompressionStream.hpp"
#include "StreamProcessor.hpp"

91
Include/Aurora/Compression/CompressionInfo.hpp
View File

@ -13,38 +13,93 @@ namespace Aurora::Compression
{
ECompresionType type;
/// ZSTD: -5 <= level <= 22
/// recommended: ZSTD_CLEVEL_DEFAULT
/// LZMA: 0 <= level <= 9
/// LZ4 : N/A
/// ZLIB: 0 <= x >= 9
/// recommended: 6
/// BZIP: 0 <= x >= 9
AuInt8 compressionLevel{};
/**
* @brief
*
* ZSTD: -5 <= level <= 22
* recommended: ZSTD_CLEVEL_DEFAULT
* LZMA: 0 <= level <= 9
* LZ4 : N/A
* Deflate: 0 <= x >= 9,
* recommended: 6
* Zip: 0 <= x >= 9,
* recommended: 6
* GZip: 0 <= x >= 9,
* recommended: 6
* BZIP: 0 <= x >= 9
*
*/
AuInt8 compressionLevel {6};
/// LZMA: 5 <= fb <= 273, default = 32
AuUInt32 fbWordSize{};
/**
* @brief LZMA: 5 <= fb <= 273, default = 32
*/
AuUInt32 fbWordSize {32};
/// LZMA only
/// (1 << 12) <= dictSize <= (1 << 27) for 32-bit version
/// (1 << 12) <= dictSize <= (3 << 29) for 64-bit version
/// default = (1 << 24)
/**
* LZMA only
* (1 << 12) <= dictSize <= (1 << 27) for 32-bit version
* (1 << 12) <= dictSize <= (3 << 29) for 64-bit version
* default = (1 << 24)
*/
AuUInt32 dictSize{};
// 64KiB is a recommended "small" block size
/**
* @brief 64KiB is a recommended "small" block size
*/
AuUInt16 lz4BlockSize {};
};
/**
* @brief DEFLATE related variabl
*
* Deflate: 0 <= x >= 15,
* recommended: 15
* Zip: 0 <= x >= 15,
* recommended: 15
* GZip: 0 <= x >= 15,
* recommended: 15
*/
AuUInt8 windowBits {15};
/**
* @brief Internal output buffer size.
* Internal swap page is undefined.
*/
AuUInt32 internalStreamSize {4096};
AuUInt8 threads {1};
; };
struct DecompressInfo
{
/**
* @brief algorithm
*/
ECompresionType alg {ECompresionType::eDeflate};
/**
* @brief Internal output buffer size. Internal swap page is undefined.
*/
AuUInt32 internalStreamSize {4096};
/**
* @brief Flag for headerless decompression streams
*/
bool hasWindowbits {true};
/**
* @brief Flag for headerless decompression streams
*/
AuInt8 windowBits {15};
DecompressInfo(ECompresionType alg) : alg(alg)
{
}
AuUInt32 internalStreamSize {};
bool permitResize {};
DecompressInfo(ECompresionType alg, AuUInt32 bufferSize) : alg(alg), internalStreamSize(bufferSize)
{
}
};
}

8
Include/Aurora/Compression/ECompresionType.hpp
View File

@ -9,12 +9,14 @@
namespace Aurora::Compression
{
enum class ECompresionType
{
AUE_DEFINE(ECompresionType,
(
eLZMA,
eZSTD,
eDeflate,
eZip,
eGZip,
eLZ4,
eBZIP2
};
));
}

61
Include/Aurora/Compression/ICompressionStream.hpp
View File

@ -9,25 +9,66 @@
namespace Aurora::Compression
{
class ICompressionStream
struct ICompressionStream
{
public:
/// Ingest n bytes from the input stream assigned to the compression object
/**
* @brief Ingest n bytes from the input stream assigned to the compression object.
* On error, returns {0, 0} or {bytesRead, 0}
* If the stream buffer runs out of memory, {bytesRead, 0} is expected, and although
* `GetAvailableProcessedBytes()` will still return some data, the decompressed data
* or uncompressed stream will be dropped in part, and you should destroy from the stream object
* @param bytesFromUnprocessedInputSource
* @return
*/
virtual AuStreamReadWrittenPair_t Ingest(AuUInt32 bytesFromUnprocessedInputSource) = 0;
/// Limited stream API
virtual bool ReadByProcessedN (void * /*opt*/, AuUInt32 minimumProcessed, AuStreamReadWrittenPair_t &pair, bool ingestUntilEOS = true) = 0;
/**
* @brief Returns the available bytes for immediate release
* @return
*/
virtual AuUInt32 GetAvailableProcessedBytes() = 0;
/// Limited stream API
virtual bool ReadByProcessedN (void * /*opt*/, AuUInt32 minimumProcessed) = 0;
/**
* @brief Returns the internal overhead to store the seekable stream buffer
* @return
*/
virtual AuUInt32 GetInternalBufferSize() = 0;
/// Limited stream API
/**
* @brief Reads 'minimumProcessed', optionally into the first buffer, until EOS or destination length.
* If the destination is null and the length is a nonzero value, the stream seeks ahead
* If the destination is null and the length is a zero, {0, GetAvailableProcessedBytes} is returned
* @param destination
* @param ingestUntilEOS should continue to poll Ingest with an arbitrary page size to fulfill destination.length
* @return Bytes read / written
*/
virtual AuStreamReadWrittenPair_t ReadEx(const Memory::MemoryViewWrite & /*opt*/ destination, bool ingestUntilEOS = true) = 0;
/**
* @brief Reads 'minimumProcessed', optionally into the first buffer, from the internal stream buffer
* @param destination
* @return Bytes written
*/
virtual AuUInt32 Read(const Memory::MemoryViewWrite & /*opt*/ destination) = 0;
/**
* @brief Seek processed read functions backwards
* @param offset
* @return
*/
virtual bool GoBackByProcessedN (AuUInt32 offset) = 0;
/// Limited stream API
/**
* @brief Seek read processed forward
* @param offset
* @return
*/
virtual bool GoForwardByProcessedN(AuUInt32 offset) = 0;
/// Compression only
virtual void Flush() = 0;
virtual bool Flush() = 0;
/// Compression only
virtual bool Finish() = 0;
};
}

2
Include/Aurora/Compression/StreamPipeProcessor.hpp
View File

@ -14,7 +14,7 @@ namespace Aurora::Compression
/// algorithm
/// LZMA decompression + compression, and ZSTD compression only
AuUInt32 threads;
AuUInt32 threads {1};
/// consume from stream callback
AuFunction<AuUInt(void *, AuUInt)> inPipe;

7
Include/Aurora/Compression/StreamProcessor.hpp
View File

@ -11,6 +11,13 @@
namespace Aurora::Compression
{
/**
* @brief
*/
AUKN_SHARED_API(Decompressor, ICompressionStream, const AuSPtr<IO::IStreamReader> &reader, const DecompressInfo &info);
/**
* @brief
*/
AUKN_SHARED_API(Compressor, ICompressionStream, const AuSPtr<IO::IStreamReader> &reader, const CompressionInfo &info);
}

3
Include/Aurora/Console/Commands/Commands.hpp
View File

@ -8,6 +8,7 @@
#pragma once
#include <Aurora/Parse/Parse.hpp>
#include "ICommandSubscriber.hpp"
namespace Aurora::Async
{
@ -15,8 +16,6 @@ namespace Aurora::Async
struct WorkerPId_t;
}
#include "ICommandSubscriber.hpp"
namespace Aurora::Console::Commands
{
AUKN_SYM void AddCommand(const AuString &tag, const Parse::ParseObject &commandStructure, const AuSPtr<ICommandSubscriber> &subscriber);

21
Include/Aurora/Console/Console.hpp
View File

@ -10,23 +10,13 @@
#include "EAnsiColor.hpp"
#include "ConsoleMessage.hpp"
namespace Aurora::Logging
{
struct ILogger;
}
namespace Aurora::Console
{
namespace Logging
{
struct ILogger;
}
/// Writes a log message to the console subscribers and telemetry outputs
AUKN_SYM void WriteLine(AuUInt8 level, const ConsoleMessage &msg);
/**
* @brief Overloads the ILogger backend of the AuLogXX functions
* @param defaultGlobalLogger
* @return
*/
AUKN_SYM void SetGlobalLogger(const AuSPtr<Logging::ILogger> &defaultGlobalLogger);
/**
* @brief Returns the untouched ILogger interface of the AuLogXX functions as configured by the Aurora::RuntimeStartInfo structure
* @return
@ -63,4 +53,3 @@ namespace Aurora::Console
#include "Commands/Commands.hpp"
#include "Hooks/Hooks.hpp"
#include "Logging/Logging.hpp"

3
Include/Aurora/Console/ConsoleMessage.hpp
View File

@ -47,6 +47,9 @@ namespace Aurora::Console
tid = Threading::Threads::GetThreadId();
}
AUKN_SYM void Read(Memory::ByteBuffer &deserialize);
AUKN_SYM void Write(Memory::ByteBuffer &serialize) const;
AUKN_SYM AuString StringifyTime(bool simple = false) const;
AUKN_SYM AuString StringifyTimeUTC() const;
AUKN_SYM AuString GetWrappedTag() const;

9
Include/Aurora/Console/EAnsiColor.hpp
View File

@ -9,8 +9,8 @@
namespace Aurora::Console
{
enum class EAnsiColor
{
AUE_DEFINE(EAnsiColor,
(
eRed,
eBoldRed,
eGreen,
@ -23,7 +23,6 @@ namespace Aurora::Console
eBoldMagenta,
eCyan,
eBoldCyan,
eReset,
eCount
};
eReset
));
}

17
Include/Aurora/Console/Logging/IBasicSink.hpp
View File

@ -1,17 +0,0 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IBasicSink.hpp
Date: 2021-11-1
Author: Reece
***/
#pragma once
namespace Aurora::Console::Logging
{
AUKN_INTERFACE(IBasicSink,
AUI_METHOD(void, OnMessageBlocking, (AuUInt8, level, const ConsoleMessage &, msg)),
AUI_METHOD(void, OnMessageNonblocking, (AuUInt8, level, const ConsoleMessage &, msg)),
AUI_METHOD(void, OnFlush, ())
)
}

17
Include/Aurora/Console/Logging/IBasicSinkRB.hpp
View File

@ -1,17 +0,0 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IBasicSinkRB.hpp
Date: 2021-11-2
Author: Reece
***/
#pragma once
namespace Aurora::Console::Logging
{
struct IBasicSinkRB : IBasicSink
{
virtual void SaveToPath(const AuString &path, bool plainText = false) = 0;
virtual AuList<ConsoleMessage> Export() = 0;
};
}

23
Include/Aurora/Console/Logging/Sinks.hpp
View File

@ -1,23 +0,0 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: Sinks.hpp
Date: 2021-11-2
Author: Reece
***/
#pragma once
namespace Aurora
{
struct SocketConsole;
}
namespace Aurora::Console::Logging
{
AUKN_SHARED_API(NewStdSink, IBasicSink);
AUKN_SHARED_API(NewOSEventDirectorySink, IBasicSink);
AUKN_SHARED_API(NewFileSink, IBasicSink, const AuString &path, bool binary = false);
AUKN_SHARED_API(NewIPCSink, IBasicSink, const SocketConsole &console);
AUKN_SHARED_API(NewRingLogger, IBasicSinkRB, AuUInt32 approxMaxBytes);
AUKN_SHARED_API(NewLogger, ILogger, const AuList<AuSPtr<IBasicSink>> &sinks);
}

6
Include/Aurora/Crypto/ECC/EECCCurve.hpp
View File

@ -9,12 +9,12 @@
namespace Aurora::Crypto::ECC
{
enum EECCCurve
{
AUE_DEFINE(EECCCurve,
(
eCurve256,
eCurve384,
eCurve521,
eCurveX25519,
eCurveEd25519
};
));
}

93
Include/Aurora/Data/TypedValue.hpp
View File

@ -13,5 +13,98 @@ namespace Aurora::Data
{
EDataType type;
Value value;
TypedValue()
{
type = EDataType::kTypeEND;
}
#define TYPEDVALUE_CONSTRUCTOR(typen, typeval, datatype) inline TypedValue(typen typeval) : value(typeval), type(datatype) {}
TYPEDVALUE_CONSTRUCTOR(const AuVec3 &, vec3, EDataType::kTypeVec3);
TYPEDVALUE_CONSTRUCTOR(const AuVec4 &, vec4, EDataType::kTypeVec4);
TYPEDVALUE_CONSTRUCTOR(bool, boolean, EDataType::kTypeBoolean);
TYPEDVALUE_CONSTRUCTOR(double, number, EDataType::kTypeNumber);
TYPEDVALUE_CONSTRUCTOR(AuInt64, sint, EDataType::kTypeSInt);
TYPEDVALUE_CONSTRUCTOR(AuUInt64, uint, EDataType::kTypeUInt);
TYPEDVALUE_CONSTRUCTOR(uuids::uuid, uuid, EDataType::kTypeUUID);
TYPEDVALUE_CONSTRUCTOR(const AuString &, str, EDataType::kTypeString);
#undef TYPEDVALUE_CONSTRUCTOR
inline AuString ToString() const
{
if (!IsValid()) return "INVALID DATATYPE";
switch (type)
{
case EDataType::kTypeUInt:
return AuToString(value.primitive.uint);
case EDataType::kTypeSInt:
return AuToString(value.primitive.sint);
case EDataType::kTypeNumber:
return AuToString(value.primitive.number);
case EDataType::kTypeString:
return value.string;
case EDataType::kTypeBoolean:
return value.primitive.boolean ? "true" : "false";
case EDataType::kTypeUUID:
return uuids::to_string(value.UUID);
case EDataType::kTypeVec3:
return AuToString(value.primitive.vec3[0]) + "." +
AuToString(value.primitive.vec3[1]) + "." +
AuToString(value.primitive.vec3[2]);
case EDataType::kTypeVec4:
return AuToString(value.primitive.vec4[0]) + "." +
AuToString(value.primitive.vec4[1]) + "." +
AuToString(value.primitive.vec4[2]) + "." +
AuToString(value.primitive.vec4[3]);
default:
return {};
}
return {};
}
inline bool operator==(const TypedValue &cmp) const
{
if (cmp.type != type)
{
return false;
}
switch (type)
{
case EDataType::kTypeUInt:
return cmp.value.primitive.uint == value.primitive.uint;
case EDataType::kTypeSInt:
return cmp.value.primitive.sint == value.primitive.sint;
case EDataType::kTypeNumber:
return cmp.value.primitive.number == value.primitive.number;
case EDataType::kTypeString:
return cmp.value.string == value.string;
case EDataType::kTypeBoolean:
return cmp.value.primitive.boolean == value.primitive.boolean;
case EDataType::kTypeUUID:
return cmp.value.UUID == value.UUID;
case EDataType::kTypeVec3:
return cmp.value.primitive.vec3 == value.primitive.vec3;
case EDataType::kTypeVec4:
return cmp.value.primitive.vec4 == value.primitive.vec4;
default:
return false;
}
return false;
}
inline bool IsValid() const
{
return static_cast<int>(type) < static_cast<int>(EDataType::kTypeGenericMax);
}
inline operator bool() const
{
return IsValid();
}
};
}

18
Include/Aurora/Debug/Debug.hpp
View File

@ -48,6 +48,24 @@ namespace Aurora::Debug
AUKN_SYM void CheckErrors();
/**
* @brief Specifies an EFailureCategory value for the current thread
* @param category
* @return
*/
AUKN_SYM void ErrorCategorySet(EFailureCategory category);
/**
* @brief Guarantees a refresh of ErrorCategoryGet's current value
* @return
*/
AUKN_SYM void ErrorCategoryClear();
/**
* @brief Returns the last EFailureCategory as specified by ErrorCategorySet or SysPushErrors
* @return
*/
AUKN_SYM EFailureCategory ErrorCategoryGet();
AUKN_SYM StackTrace GetStackTrace();

10
Include/Aurora/Debug/EFailureCategory.hpp
View File

@ -34,7 +34,17 @@ namespace Aurora::Debug
kFailureDisconnected,
kFailureUninitialized,
kFailureUnimplemented,
kFailureCatch,
kFailureSubmission,
kFailureLockError,
kFailureNoAccess,
kFailureResourceMissing,
kFailureResourceLocked,
kFailureMalformedData,
kFailureInSandboxContext,
kFailureParseError,
kFailureNone = 255,
kFailureUserBegin = 256
};
}

16
Include/Aurora/Debug/SysAssertions.hpp
View File

@ -21,8 +21,8 @@
return;
}
Aurora::Console::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Console::Logging::ELogLevel::eError),
Aurora::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Logging::ELogLevel::eError),
Aurora::Console::EAnsiColor::eBoldRed,
"Fatal",
"Expression address: {} {}:{}", func, file, fileno);
@ -107,14 +107,14 @@
return;
}
Aurora::Console::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Console::Logging::ELogLevel::eError),
Aurora::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Logging::ELogLevel::eError),
Aurora::Console::EAnsiColor::eBoldRed,
"Fatal",
"Expression address: {} {}:{}", func, file, fileno);
Aurora::Console::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Console::Logging::ELogLevel::eError),
Aurora::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Logging::ELogLevel::eError),
Aurora::Console::EAnsiColor::eBoldRed,
"Fatal",
"Expression failed: {}", exp);
@ -158,8 +158,8 @@
return;
}
Aurora::Console::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Console::Logging::ELogLevel::eError),
Aurora::Logging::WriteLinef(
static_cast<AuUInt8>(Aurora::Logging::ELogLevel::eError),
Aurora::Console::EAnsiColor::eBoldRed,
"Fatal",
"Expression failed: {}", exp);

97
Include/Aurora/Debug/SysErrors.hpp
View File

@ -28,23 +28,55 @@ namespace Aurora::Debug
#define _FREECOMPILER_OPTIMIZE_OFF __attribute__((optimize("0")))
#endif
static AU_NOINLINE void ErrorMakeNested() _FREECOMPILER_OPTIMIZE_OFF
// TODO: bring the xenus thing into here
static AU_NOINLINE AuUInt GetIPNoBackend() _FREECOMPILER_OPTIMIZE_OFF
{
return _DBG_RET_ADDR;
}
#if defined(AURORA_COMPILER_MSVC)
#pragma optimize("", on)
#endif
#undef _FREECOMPILER_OPTIMIZE_OFF
static auline void ErrorMakeNested()
{
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, nullptr);
}
template<typename ... T>
static AU_NOINLINE void ErrorMakeNested(const AuString &msg, T&& ... args) _FREECOMPILER_OPTIMIZE_OFF
auline void ErrorMakeNested(const AuString &msg, T&& ... args)
{
if constexpr (sizeof...(T) == 0)
{
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, msg.c_str());
}
else
{
#if defined(_AUHAS_FMT)
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, fmt::format(msg, AuForward<T>(args)...).c_str());
try
{
auto tempString = fmt::format(msg, AuForward<T>(args)...);
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, tempString.c_str());
}
catch (...)
{
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, msg.c_str());
}
#else
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, nullptr);
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, msg.c_str());
#endif
}
}
static auline void ErrorMakeNested(const char *msg)
{
_PushError(_DBG_RET_ADDR, EFailureCategory::kFailureNested, msg);
}
template<typename ... T>
static AU_NOINLINE void SysPushError(EFailureCategory category, const AuString &msg, T&& ... args) _FREECOMPILER_OPTIMIZE_OFF
auline void SysPushError(EFailureCategory category, const AuString &msg, T&& ... args)
{
if constexpr (sizeof...(T) == 0)
{
@ -53,22 +85,30 @@ namespace Aurora::Debug
else
{
#if defined(_AUHAS_FMT)
_PushError(_DBG_RET_ADDR, category, fmt::format(msg, AuForward<T>(args)...).c_str());
try
{
auto tempString = fmt::format(msg, AuForward<T>(args)...);
_PushError(_DBG_RET_ADDR, category, tempString.c_str());
}
catch (...)
{
_PushError(_DBG_RET_ADDR, category, msg.c_str());
}
#else
_PushError(_DBG_RET_ADDR, category, "Missing dependency");
_PushError(_DBG_RET_ADDR, category, msg.c_str());
#endif
}
}
static AU_NOINLINE void SysPushError(EFailureCategory category) _FREECOMPILER_OPTIMIZE_OFF
static auline void SysPushError(EFailureCategory category, const char *msg)
{
_PushError(_DBG_RET_ADDR, category, msg);
}
static auline void SysPushError(EFailureCategory category)
{
_PushError(_DBG_RET_ADDR, category, nullptr);
}
#if defined(AURORA_COMPILER_MSVC)
#pragma optimize("", on)
#endif
#undef _FREECOMPILER_OPTIMIZE_OFF
}
#define SysCheckReturn(x, ...) if (!(static_cast<bool>(x))) { Aurora::Debug::ErrorMakeNested(); return __VA_ARGS__; }
@ -88,6 +128,7 @@ namespace Aurora::Debug
// enums
#define SysPushErrorGeneric(...) SysPushErrorError(kFailureGeneric, ## __VA_ARGS__)
#define SysPushErrorCatch(...) SysPushErrorError(kFailureCatch, ## __VA_ARGS__)
#define SysPushErrorMemory(...) SysPushErrorError(kFailureMemory, ## __VA_ARGS__)
#define SysPushErrorIO(...) SysPushErrorError(kFailureIO, ## __VA_ARGS__)
#define SysPushErrorFIO(...) SysPushErrorError(kFailureFIO, ## __VA_ARGS__)
@ -109,6 +150,15 @@ namespace Aurora::Debug
#define SysPushErrorDisconnected(...) SysPushErrorError(kFailureDisconnected, ## __VA_ARGS__)
#define SysPushErrorUninitialized(...) SysPushErrorError(kFailureUninitialized, ## __VA_ARGS__)
#define SysPushErrorUnimplemented(...) SysPushErrorError(kFailureUnimplemented, ## __VA_ARGS__)
#define SysPushErrorSubmission(...) SysPushErrorError(kFailureSubmission, ## __VA_ARGS__)
#define SysPushErrorLockError(...) SysPushErrorError(kFailureLockError, ## __VA_ARGS__)
#define SysPushErrorSyntax(...) SysPushErrorError(kFailureSyntax, ## __VA_ARGS__)
#define SysPushErrorNoAccess(...) SysPushErrorError(kFailureNoAccess, ## __VA_ARGS__)
#define SysPushErrorResourceMissing(...) SysPushErrorError(kFailureResourceMissing, ## __VA_ARGS__)
#define SysPushErrorResourceLocked(...) SysPushErrorError(kFailureResourceLocked, ## __VA_ARGS__)
#define SysPushErrorMalformedData(...) SysPushErrorError(kFailureMalformedData, ## __VA_ARGS__)
#define SysPushErrorInSandboxContext(...) SysPushErrorError(kFailureInSandboxContext, ## __VA_ARGS__)
#define SysPushErrorParseError(...) SysPushErrorError(kFailureParseError, ## __VA_ARGS__)
#if defined(DEBUG) || defined(STAGING)
@ -144,6 +194,16 @@ namespace Aurora::Debug
#define SysPushErrorDisconnectedDbg SysPushErrorDisconnected
#define SysPushErrorUninitializedDbg SysPushErrorUninitialized
#define SysPushErrorUnimplementedDbg SysPushErrorUnimplemented
#define SysPushErrorCatchDbg SysPushErrorCatch
#define SysPushErrorSubmissionDbg SysPushErrorSubmission
#define SysPushErrorLockErrorDbg SysPushErrorLockError
#define SysPushErrorSyntaxDbg SysPushErrorSyntax
#define SysPushErrorNoAccessDbg SysPushErrorNoAccess
#define SysPushErrorResourceMissingDbg SysPushErrorResourceMissing
#define SysPushErrorResourceLockedDbg SysPushErrorResourceLocked
#define SysPushErrorMalformedDataDbg SysPushErrorMalformedData
#define SysPushErrorInSandboxContextDbg SysPushErrorInSandboxContext
#define SysPushErrorParseErrorDbg SysPushErrorParseError
#else
@ -177,5 +237,16 @@ namespace Aurora::Debug
#define SysPushErrorDisconnectedDbg(...)
#define SysPushErrorUninitializedDbg(...)
#define SysPushErrorUnimplementedDbg(...)
#define SysPushErrorCatchDbg(...)
#define SysPushErrorSubmissionDbg(...)
#define SysPushErrorLockErrorDbg(...)
#define SysPushErrorSyntaxDbg(...)
#define SysPushErrorNoAccessDbg(...)
#define SysPushErrorResourceMissingDbg(...)
#define SysPushErrorResourceLockedDbg(...)
#define SysPushErrorMalformedDataDbg(...)
#define SysPushErrorInSandboxContextDbg(...)
#define SysPushErrorParseErrorDbg(...)
#endif

4
Include/Aurora/Debug/SysPanic.hpp
View File

@ -9,9 +9,9 @@
#if defined(_AUHAS_FMT)
template<typename ... T>
static inline void __declspec(noreturn) SysPanic(T... args)
static inline void AU_NORETURN SysPanic(T... args)
{
Aurora::Console::Logging::WriteLinef(static_cast<AuUInt8>(Aurora::Console::Logging::ELogLevel::eError), Aurora::Console::EAnsiColor::eBoldRed, "Fatal", AuForward<T>(args)...);
Aurora::Logging::WriteLinef(static_cast<AuUInt8>(Aurora::Logging::ELogLevel::eError), Aurora::Console::EAnsiColor::eBoldRed, "Fatal", AuForward<T>(args)...);
Aurora::Debug::Panic();
}
#endif

26
Include/Aurora/Exit/ETriggerLevel.hpp Normal file
View File

@ -0,0 +1,26 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ETriggerLevel.hpp
Date: 2022-1-30
Author: Reece
***/
#pragma once
namespace Aurora::Exit
{
AUE_DEFINE(ETriggerLevel, (
// Runtime deinitialize
eSafeTermination,
// A fatal exception was caught by the watchdog. The process is on its way out
eFatalException,
// Control+C was sent, termination must follow
eSigTerminate,
// Something went wrong in the process. These could be frequent depending on the process
eProblematicEvent
));
}

36
Include/Aurora/Exit/Exit.hpp Normal file
View File

@ -0,0 +1,36 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: Exit.hpp
Date: 2022-1-30
Author: Reece
***/
#pragma once
#include "ETriggerLevel.hpp"
#include "IExitSubscriber.hpp"
namespace Aurora::Exit
{
/**
* @brief Registers @param callback to the registry to subscribe to @param level events
* @param level
* @param callback
* @return
*/
AUKN_SYM bool ExitHandlerAdd(ETriggerLevel level, const AuSPtr<IExitSubscriber> &callback);
/**
* @brief Removes all exit handlers by pointer
* @param callback
* @return
*/
AUKN_SYM void ExitHandlerRemove(const AuSPtr<IExitSubscriber> &callback);
/**
* @brief Used from within callbacks to determine if the application is shutting down.
* @return
*/
AUKN_SYM bool IsAppRunning();
}

23
Include/Aurora/Exit/IExitSubscriber.hpp Normal file
View File

@ -0,0 +1,23 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IExitSubscriber.hpp
Date: 2022-1-30
Author: Reece
***/
#pragma once
namespace Aurora::Exit
{
struct ExitInvoker
{
Threading::Threads::IAuroraThread *pCaller;
};
/**
* @brief User definedable callback
*/
AUKN_INTERFACE(IExitSubscriber,
AUI_METHOD(void, OnTrigger, (ETriggerLevel, level, const ExitInvoker *, pInvoker))
);
}

1
Include/Aurora/HWInfo/CpuBitId.hpp
View File

@ -32,6 +32,7 @@ namespace Aurora::HWInfo
inline CpuBitId Or(const CpuBitId &id) const;
inline bool CpuBitScanForward(AuUInt8 &index, AuUInt8 offset) const;
inline AuUInt8 CpuBitCount() const;
inline bool TestCpuIdx(AuUInt8 idx) const;
inline void Clear();

11
Include/Aurora/HWInfo/CpuBitId.inl
View File

@ -33,6 +33,17 @@ namespace Aurora::HWInfo
#endif
}
inline AuUInt8 CpuBitId::CpuBitCount() const
{
return AuPopCnt(lower) +
AuPopCnt(upper)
#if defined(_AU_MASSIVE_CPUID)
+ AuPopCnt(upper2)
+ AuPopCnt(upper3)
#endif
;
}
bool CpuBitId::CpuBitScanForward(AuUInt8 &index, AuUInt8 offset) const
{
#if defined(_AU_MASSIVE_CPUID)

54
Include/Aurora/Hashing/Digests.hpp
View File

@ -9,53 +9,9 @@
namespace Aurora::Hashing
{
AUKN_SYM void MD5(const void *buffer, AuMach length, AuArray<AuUInt8, 16> &md5);
static void MD5(const Memory::ByteBuffer &bytebuffer, AuArray<AuUInt8, 16> &md5)
{
return MD5(bytebuffer.data(), bytebuffer.size(), md5);
}
static void MD5(const AuString &bytebuffer, AuArray<AuUInt8, 16> &md5)
{
return MD5(bytebuffer.data(), bytebuffer.size(), md5);
}
AUKN_SYM void SHA1(const void *buffer, AuMach length, AuArray<AuUInt8, 20> &sha1);
static void SHA1(const Memory::ByteBuffer &bytebuffer, AuArray<AuUInt8, 20> &sha1)
{
return SHA1(bytebuffer.data(), bytebuffer.size(), sha1);
}
static void SHA1(const AuString &bytebuffer, AuArray<AuUInt8, 20> &sha1)
{
return SHA1(bytebuffer.data(), bytebuffer.size(), sha1);
}
AUKN_SYM void Tiger(const void *buffer, AuMach length, AuArray<AuUInt8, 24> &tiger);
static void Tiger(const Memory::ByteBuffer &bytebuffer, AuArray<AuUInt8, 24> &tiger)
{
return Tiger(bytebuffer.data(), bytebuffer.size(), tiger);
}
static void Tiger(const AuString &bytebuffer, AuArray<AuUInt8, 24> &tiger)
{
return Tiger(bytebuffer.data(), bytebuffer.size(), tiger);
}
AUKN_SYM void SHA2(const void *buffer, AuMach length, AuArray<AuUInt8, 32> &sha2);
static void SHA2(const Memory::ByteBuffer &bytebuffer, AuArray<AuUInt8, 32> &sha2)
{
return SHA2(bytebuffer.data(), bytebuffer.size(), sha2);
}
static void SHA2(const AuString &bytebuffer, AuArray<AuUInt8, 32> &sha2)
{
return SHA2(bytebuffer.data(), bytebuffer.size(), sha2);
}
AUKN_SYM void SHA2_64(const void *buffer, AuMach length, AuArray<AuUInt8, 64> &sha2);
static void SHA2_64(const Memory::ByteBuffer &bytebuffer, AuArray<AuUInt8, 64> &sha2)
{
return SHA2_64(bytebuffer.data(), bytebuffer.size(), sha2);
}
static void SHA2_64(const AuString &bytebuffer, AuArray<AuUInt8, 64> &sha2)
{
return SHA2_64(bytebuffer.data(), bytebuffer.size(), sha2);
}
AUKN_SYM void MD5(const Memory::MemoryViewRead &span, AuArray<AuUInt8, 16> &md5);
AUKN_SYM void SHA1(const Memory::MemoryViewRead &span, AuArray<AuUInt8, 20> &sha1);
AUKN_SYM void Tiger(const Memory::MemoryViewRead &span, AuArray<AuUInt8, 24> &tiger);
AUKN_SYM void SHA2(const Memory::MemoryViewRead &span, AuArray<AuUInt8, 32> &sha2);
AUKN_SYM void SHA2_64(const Memory::MemoryViewRead &span, AuArray<AuUInt8, 64> &sha2);
}

6
Include/Aurora/Hashing/EHashType.hpp
View File

@ -9,12 +9,12 @@
namespace Aurora::Hashing
{
enum class EHashType
{
AUE_DEFINE(EHashType,
(
eMD5,
eSHA1,
eSHA2_32,
eSHA2_64,
eTiger
};
));
}

2
Include/Aurora/Hashing/HashStream.hpp
View File

@ -15,7 +15,7 @@ namespace Aurora::Hashing
/**
* Digest length of pBuf bytes
*/
virtual void Ingest(const void *pBuf, AuUInt32 length) = 0;
virtual void Ingest(const Memory::MemoryViewRead &input) = 0;
/**
* Locks and returns the internal buffer

50
Include/Aurora/IO/Buffered/BlobArbitraryReader.hpp
View File

@ -1,50 +0,0 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: BlobArbitraryReader.hpp
Date: 2021-6-10
Author: Reece
***/
#pragma once
namespace Aurora::IO::Buffered
{
class BlobArbitraryReader : public IArbitraryStreamer
{
public:
AU_NO_COPY_NO_MOVE(BlobArbitraryReader)
BlobArbitraryReader(const Memory::ByteBuffer &buffer) : buffer_(buffer) {}
BlobArbitraryReader() {}
~BlobArbitraryReader(){}
virtual EStreamError Open() override
{
return EStreamError::eErrorNone;
}
virtual EStreamError ArbitraryRead(AuUInt32 offset, const Memory::MemoryViewStreamWrite &paramters) override
{
if (buffer_.empty()) return EStreamError::eErrorEndOfStream;
auto endOffset = offset + paramters.length;
auto realEndOffset = AuMin(buffer_.size(), endOffset);
auto actualLength = realEndOffset - offset;
if (actualLength < 0) return EStreamError::eErrorEndOfStream;
paramters.outVariable = actualLength;
AuMemcpy(paramters.ptr, buffer_.data() + offset, paramters.outVariable);
return EStreamError::eErrorNone;
}
virtual void Close() override
{
buffer_.clear();
}
private:
Memory::ByteBuffer buffer_;
};
}

0
Source/IO/Character/LineBuffer.cpp → Include/Aurora/IO/Buffered/BlobBuilder.hpp
View File

32
Include/Aurora/IO/Buffered/BlobReader.hpp
View File

@ -9,39 +9,33 @@
namespace Aurora::IO::Buffered
{
class BlobReader : public IStreamReader
struct BlobReader : public IStreamReader
{
public:
AU_NO_COPY(BlobReader)
BlobReader(const AuSPtr<Memory::ByteBuffer> &buffer) : buffer_(buffer) {}
BlobReader(Memory::ByteBuffer &&buffer) : buffer_(AuMakeShared<Memory::ByteBuffer>(buffer)) {}
BlobReader(const Memory::ByteBuffer &buffer) : buffer_(AuMakeShared<Memory::ByteBuffer>(buffer)) {}
inline BlobReader(const AuSPtr<Memory::ByteBuffer> &buffer) : buffer_(buffer) {}
inline BlobReader(Memory::ByteBuffer &&buffer) : buffer_(AuMakeShared<Memory::ByteBuffer>(buffer)) {}
inline BlobReader(const Memory::ByteBuffer &buffer) : buffer_(AuMakeShared<Memory::ByteBuffer>(buffer)) {}
BlobReader() {}
~BlobReader() {}
inline BlobReader() {}
inline ~BlobReader() {}
virtual EStreamError Open() override
inline virtual EStreamError Open() override
{
if (!buffer_) return EStreamError::eErrorStreamNotOpen;
if (!buffer_->operator bool()) return EStreamError::eErrorStreamNotOpen;
return EStreamError::eErrorNone;
}
virtual EStreamError Read(const Memory::MemoryViewStreamWrite &paramters) override
inline virtual EStreamError Read(const Memory::MemoryViewStreamWrite &parameters) override
{
auto realEndOffset = AuMin(buffer_->size() - offset_, paramters.length);
if (realEndOffset == 0) return EStreamError::eErrorEndOfStream;
paramters.outVariable = realEndOffset;
AuMemcpy(paramters.ptr, buffer_->data() + offset_, realEndOffset);
offset_ += realEndOffset;
return EStreamError::eErrorNone;
if (!buffer_) return EStreamError::eErrorStreamNotOpen;
parameters.outVariable = buffer_->Read(parameters.ptr, parameters.length);
return parameters.outVariable == 0 ? EStreamError::eErrorEndOfStream : EStreamError::eErrorNone;
}
virtual void Close() override
inline virtual void Close() override
{
if (!buffer_) return;
buffer_->clear();
}

46
Include/Aurora/IO/Buffered/BlobSeekableReader.hpp Normal file
View File

@ -0,0 +1,46 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: BlobSeekableReader.hpp
Date: 2021-6-10
Author: Reece
***/
#pragma once
namespace Aurora::IO::Buffered
{
struct BlobSeekableReader : public ISeekingReader
{
AU_NO_COPY_NO_MOVE(BlobSeekableReader)
inline BlobSeekableReader(const AuSPtr<Memory::ByteBuffer> &buffer) : buffer_(buffer) {}
inline BlobSeekableReader(Memory::ByteBuffer &&buffer) : buffer_(AuMakeShared<Memory::ByteBuffer>(buffer)) {}
inline BlobSeekableReader(const Memory::ByteBuffer &buffer) : buffer_(AuMakeShared<Memory::ByteBuffer>(buffer)) {}
inline BlobSeekableReader() {}
inline ~BlobSeekableReader(){}
inline virtual EStreamError Open() override
{
if (!buffer_) return EStreamError::eErrorStreamNotOpen;
return EStreamError::eErrorNone;
}
inline virtual EStreamError ArbitraryRead(AuUInt offset, const Memory::MemoryViewStreamWrite &parameters) override
{
if (!buffer_) return EStreamError::eErrorStreamNotOpen;
if (buffer_->empty()) return EStreamError::eErrorEndOfStream;
buffer_->readPtr = buffer_->base + offset;
parameters.outVariable = buffer_->Read(parameters.ptr, parameters.length);
return parameters.outVariable == 0 ? EStreamError::eErrorEndOfStream : EStreamError::eErrorNone;
}
inline virtual void Close() override
{
if (buffer_) buffer_->clear();
}
private:
AuSPtr<Memory::ByteBuffer> buffer_;
};
}

33
Include/Aurora/IO/Buffered/BlobWriter.hpp
View File

@ -9,41 +9,40 @@
namespace Aurora::IO::Buffered
{
class BlobWriter : public IStreamWriter
struct BlobWriter : public IStreamWriter
{
public:
AU_NO_COPY_NO_MOVE(BlobWriter)
BlobWriter() {}
~BlobWriter() {}
inline BlobWriter(const AuSPtr<Memory::ByteBuffer> &buffer) : buffer_(buffer) {}
inline BlobWriter() : buffer_(AuMakeShared<Memory::ByteBuffer>()) {}
inline ~BlobWriter() {}
virtual EStreamError Open() override
inline virtual EStreamError Open() override
{
return buffer_ ? EStreamError::eErrorNone : EStreamError::eErrorStreamNotOpen;
}
inline virtual EStreamError Write(const Memory::MemoryViewStreamRead &parameters) override
{
if (!buffer_) return EStreamError::eErrorStreamNotOpen;
parameters.outVariable = buffer_->Write(parameters.ptr, parameters.length);
return EStreamError::eErrorNone;
}
virtual EStreamError Write(const Memory::MemoryViewStreamRead & parameters) override
{
auto idx = buffer_.size();
buffer_.resize(idx + parameters.length);
AuMemcpy(buffer_.data() + idx, parameters.ptr, buffer_.size() - idx);
return EStreamError::eErrorNone;
}
virtual void Flush() override
inline virtual void Flush() override
{
}
virtual void Close() override
inline virtual void Close() override
{
}
const Memory::ByteBuffer &GetBuffer()
inline AuSPtr<Memory::ByteBuffer> GetBuffer()
{
return this->buffer_;
}
private:
Memory::ByteBuffer buffer_;
AuSPtr<Memory::ByteBuffer> buffer_;
};
}

2
Include/Aurora/IO/Buffered/Buffered.hpp
View File

@ -9,6 +9,6 @@
// Most internal transactions will be buffered.
// For sake of being able to mix network, file, serial, and other code, its nice to use the same reader interface for buffered and streams where read/write until EOS is required
#include "BlobArbitraryReader.hpp"
#include "BlobSeekableReader.hpp"
#include "BlobReader.hpp"
#include "BlobWriter.hpp"

4
Include/Aurora/IO/Character/Character.hpp
View File

@ -8,5 +8,7 @@
#pragma once
#include "ICharacterProvider.hpp"
#include "ICharacterProviderEx.hpp"
#include "IBufferedCharacterConsumer.hpp"
#include "ILineBufferer.hpp"
#include "IBufferedLineReader.hpp"
#include "Providers.hpp"

10
Include/Aurora/IO/Character/IBufferedCharacterConsumer.hpp
View File

@ -9,5 +9,15 @@
namespace Aurora::IO::Character
{
struct IBufferedCharacterConsumer
{
virtual bool HasBufferedNext() const = 0;
virtual AuUInt8 NextBufferedByte() const = 0;
virtual bool PeekNext(AuUInt8 &out) = 0;
virtual AuUInt8 GetCurrent() = 0;
virtual bool Next(AuUInt8 &out) = 0;
};
AUKN_SHARED_API(BufferConsumerFromProvider, IBufferedCharacterConsumer, const AuSPtr<ICharacterProvider> &provider);
}

20
Include/Aurora/IO/Character/IBufferedLineReader.hpp Normal file
View File

@ -0,0 +1,20 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IBufferedLineReader.hpp
Date: 2022-1-29
Author: Reece
***/
#pragma once
namespace Aurora::IO::Character
{
struct IBufferedLineReader
{
virtual bool ReadBytes(AuUInt32 length) = 0;
virtual void Flush() = 0;
virtual AuList<AuString> ReadLines() = 0;
};
AUKN_SHARED_API(NewLineReader, IBufferedLineReader, const AuSPtr<IStreamReader> &input);
}

2
Include/Aurora/IO/Character/ICharacterProvider.hpp
View File

@ -10,6 +10,6 @@
namespace Aurora::IO::Character
{
AUKN_INTERFACE(ICharacterProvider,
AUI_METHOD(AuUInt8, GetByte, ())
AUI_METHOD(bool, GetByte, (AuUInt8 &, val))
);
}

17
Include/Aurora/IO/Character/ICharacterProviderEx.hpp Normal file
View File

@ -0,0 +1,17 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ICharacterProviderEx.hpp
Date: 2022-1-29
Author: Reece
***/
#pragma once
namespace Aurora::IO::Character
{
struct ICharacterProviderEx : ICharacterProvider
{
virtual AuUInt GetPosition() = 0;
virtual bool SetPosition(AuUInt offset) = 0;
};
}

16
Include/Aurora/IO/Character/Providers.hpp Normal file
View File

@ -0,0 +1,16 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: Providers.hpp
Date: 2022-1-29
Author: Reece
***/
#pragma once
namespace Aurora::IO::Character
{
// you are responsible for maintaining a lock over shared string writes and provider use instances
// (string may change so long as a read/write lock is protecting ICharacterProviderEx reads)
AUKN_SHARED_API(ProviderFromSharedString, ICharacterProviderEx, const AuSPtr<AuString> &str, AuUInt index = 0);
AUKN_SHARED_API(ProviderFromString, ICharacterProviderEx, const AuString &str, AuUInt index = 0);
}

5
Include/Aurora/IO/EStreamError.hpp
View File

@ -9,12 +9,11 @@
namespace Aurora::IO
{
enum class EStreamError
{
AUE_DEFINE_VA(EStreamError,
eErrorNone,
eErrorEndOfStream,
eErrorStreamNotOpen,
eErrorStreamInterrupted,
eErrorHandleClosed
};
);
}

31
Include/Aurora/IO/FS/Async.hpp
View File

@ -9,36 +9,7 @@
namespace Aurora::IO::FS
{
class IAsyncTransaction;
class IAsyncFileStream
{
public:
virtual AuSPtr<IAsyncTransaction> NewTransaction() = 0;
};
class IAsyncFinishedSubscriber
{
public:
virtual void OnAsyncFileOpFinished(AuUInt64 offset, AuUInt32 length) = 0;
};
class IAsyncTransaction
{
public:
// Do not switch to Aurora::Memory::MemoryView, you must use a raw pointer to the parent object that effectively owns the IAsyncTransaction
virtual bool StartRead(AuUInt64 offset, void *, AuUInt32 length) = 0;
virtual bool StartWrite(AuUInt64 offset, const void *, AuUInt32 length) = 0;
virtual bool Complete() = 0;
virtual AuUInt32 GetLastPacketLength() = 0;
virtual void SetCallback(const AuSPtr<IAsyncFinishedSubscriber> &sub) = 0;
virtual bool Wait(AuUInt32 timeout) = 0;
};
AUKN_SHARED_API(OpenAsync, IAsyncFileStream, const AuString &path, bool readOnly = true, bool directIO = false);
AUKN_SHARED_API(OpenAsync, IAsyncFileStream, const AuString &path, EFileOpenMode openMode, bool directIO = false, EFileAdvisoryLockLevel lock = EFileAdvisoryLockLevel::eNoSafety);
/// \param transactions Array of FIO transactions
/// \param timeout Aurora Timeout

29
Include/Aurora/IO/FS/EFileAdvisoryLockLevel.hpp Normal file
View File

@ -0,0 +1,29 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: EFileAdvisoryLockLevel.hpp
Date: 2022-1-29
Author: Reece
***/
#pragma once
namespace Aurora::IO::FS
{
// I know NT can do much better, network drives and unix can not.
// Do not open an issue over this
// There is no portable solution for file locks
AUE_DEFINE(EFileAdvisoryLockLevel,
(
// Nothing will invalidate the file stream, potentially causing corruption on lock race
// No lock will be placed on the file nor created to be polled
// File locks just aren't portable enough yet
eNoSafety,
// Prevent software that is willing to play ball from writing to the file path
eBlockWrite,
// Prevent software that is willing to play ball from reading the file, write blocking inherited
eBlockReadWrite
));
}

18
Include/Aurora/IO/FS/EFileOpenMode.hpp Normal file
View File

@ -0,0 +1,18 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: EOpenMode.hpp
Date: 2022-1-29
Author: Reece
***/
#pragma once
namespace Aurora::IO::FS
{
AUE_DEFINE(EFileOpenMode,
(
eRead,
eWrite,
eReadWrite
));
}

7
Include/Aurora/IO/FS/FS.hpp
View File

@ -44,11 +44,16 @@ namespace Aurora::IO::FS
AUKN_SYM bool GoUpToSeparator(AuString &out, const AuString &path);
}
#include "EFileAdvisoryLockLevel.hpp"
#include "EFileOpenMode.hpp"
#include "IFileStream.hpp"
#include "FileStream.hpp"
#include "FileArbitraryReader.hpp"
#include "FileSeekableReader.hpp"
#include "FileReader.hpp"
#include "FileWriter.hpp"
#include "Resources.hpp"
#include "Stat.hpp"
#include "IAsyncFileStream.hpp"
#include "IAsyncFinishedSubscriber.hpp"
#include "IAsyncTransaction.hpp"
#include "Async.hpp"

15
Include/Aurora/IO/FS/FileReader.hpp
View File

@ -9,27 +9,26 @@
namespace Aurora::IO::FS
{
class FileReader : public IStreamReader
struct FileReader : public IStreamReader
{
public:
AU_NO_COPY_NO_MOVE(FileReader)
FileReader() {}
~FileReader() {}
inline FileReader() {}
inline ~FileReader() {}
template<typename... T>
bool OpenFile(T... args)
inline bool OpenFile(T... args)
{
stream_ = OpenReadUnique(args...);
return stream_ != nullptr;
}
virtual EStreamError Open() override
inline virtual EStreamError Open() override
{
return stream_ ? EStreamError::eErrorNone : EStreamError::eErrorStreamNotOpen;
}
virtual EStreamError Read(const Memory::MemoryViewStreamWrite &paramters) override
inline virtual EStreamError Read(const Memory::MemoryViewStreamWrite &paramters) override
{
if (!stream_) return EStreamError::eErrorStreamNotOpen;
if (!stream_->Read(paramters)) return EStreamError::eErrorStreamInterrupted;
@ -37,7 +36,7 @@ namespace Aurora::IO::FS
return EStreamError::eErrorNone;
}
virtual void Close() override
inline virtual void Close() override
{
stream_.reset();
}

13
Include/Aurora/IO/FS/FileArbitraryReader.hpp → Include/Aurora/IO/FS/FileSeekableReader.hpp
View File

@ -1,7 +1,7 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: FileArbitraryReader.hpp
File: FileSeekableReader.hpp
Date: 2021-6-10
Author: Reece
***/
@ -9,13 +9,12 @@
namespace Aurora::IO::FS
{
class FileArbitraryReader : public IArbitraryStreamer
struct FileSeekableReader : public ISeekingReader
{
public:
AU_NO_COPY_NO_MOVE(FileArbitraryReader)
AU_NO_COPY_NO_MOVE(FileSeekableReader)
FileArbitraryReader() {}
~FileArbitraryReader() {}
FileSeekableReader() {}
~FileSeekableReader() {}
template<typename... T>
bool OpenFile(T... args)
@ -29,7 +28,7 @@ namespace Aurora::IO::FS
return stream_ ? EStreamError::eErrorNone : EStreamError::eErrorStreamInterrupted;
}
virtual EStreamError ArbitraryRead(AuUInt32 offset, const Memory::MemoryViewStreamWrite &paramters) override
virtual EStreamError ArbitraryRead(AuUInt offset, const Memory::MemoryViewStreamWrite &paramters) override
{
if (!stream_) return EStreamError::eErrorStreamNotOpen;
if (!stream_->SetOffset(offset)) return EStreamError::eErrorEndOfStream;

6
Include/Aurora/IO/FS/FileStream.hpp
View File

@ -9,6 +9,8 @@
namespace Aurora::IO::FS
{
AUKN_SHARED_API(OpenRead, IFileStream, const AuString &path);
AUKN_SHARED_API(OpenWrite, IFileStream, const AuString &path);
AUKN_SHARED_API(OpenRead, IFileStream, const AuString &path, EFileAdvisoryLockLevel successPostAdvisoryLevel = EFileAdvisoryLockLevel::eBlockWrite);
AUKN_SHARED_API(OpenWrite, IFileStream, const AuString &path, EFileAdvisoryLockLevel successPostAdvisoryLevel = EFileAdvisoryLockLevel::eBlockReadWrite);
AUKN_SHARED_API(Open, IFileStream, const AuString &path, EFileOpenMode mode = EFileOpenMode::eRead, EFileAdvisoryLockLevel successPostAdvisoryLevel = EFileAdvisoryLockLevel::eBlockReadWrite);
}

27
Include/Aurora/IO/FS/FileWriter.hpp
View File

@ -9,27 +9,26 @@
namespace Aurora::IO::FS
{
class FileWriter : public IStreamWriter
struct FileWriter : public IStreamWriter
{
public:
AU_NO_COPY_NO_MOVE(FileWriter)
FileWriter() {}
~FileWriter() {}
inline FileWriter() {}
inline ~FileWriter() {}
template<typename... T>
bool OpenFile(T... args)
inline bool OpenFile(T... args)
{
stream_ = OpenWriteUnique(args...);
return stream_ != nullptr;
}
virtual EStreamError Open() override
inline virtual EStreamError Open() override
{
return stream_ ? EStreamError::eErrorNone : EStreamError::eErrorStreamNotOpen;
}
virtual EStreamError Write(const Memory::MemoryViewStreamRead & parameters) override
inline virtual EStreamError Write(const Memory::MemoryViewStreamRead &parameters) override
{
if (!stream_) return EStreamError::eErrorStreamNotOpen;
if (!stream_->Write(parameters)) return EStreamError::eErrorStreamInterrupted;
@ -38,16 +37,26 @@ namespace Aurora::IO::FS
return EStreamError::eErrorNone;
}
virtual void Flush() override
inline virtual void Flush() override
{
if (stream_) stream_->Flush();
}
virtual void Close() override
inline virtual void Close() override
{
stream_.reset();
}
inline virtual AuUInt64 GetPosition()
{
return stream_ ? stream_->GetOffset() : 0;
}
inline virtual bool SetPosition(AuUInt64 pos)
{
return stream_ ? stream_->SetOffset(pos) : false;
}
private:
OpenWriteUnique_t stream_{};
};

18
Include/Aurora/IO/FS/IAsyncFileStream.hpp Normal file
View File

@ -0,0 +1,18 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IAsyncFileStream.hpp
Date: 2022-2-14
Author: Reece
***/
#pragma once
namespace Aurora::IO::FS
{
struct IAsyncTransaction;
struct IAsyncFileStream
{
virtual AuSPtr<IAsyncTransaction> NewTransaction() = 0;
};
}

15
Include/Aurora/IO/FS/IAsyncFinishedSubscriber.hpp Normal file
View File

@ -0,0 +1,15 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IAsyncFinishedSubscriber.hpp
Date: 2022-2-14
Author: Reece
***/
#pragma once
namespace Aurora::IO::FS
{
AUKN_INTERFACE(IAsyncFinishedSubscriber,
AUI_METHOD(void, OnAsyncFileOpFinished, (AuUInt64, offset, AuUInt32, length))
);
}

30
Include/Aurora/IO/FS/IAsyncTransaction.hpp Normal file
View File

@ -0,0 +1,30 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IAsyncTransaction.hpp
Date: 2022-2-14
Author: Reece
***/
#pragma once
namespace Aurora::Loop
{
struct ILoopSource;
}
namespace Aurora::IO::FS
{
struct IAsyncTransaction
{
virtual bool StartRead(AuUInt64 offset, const AuSPtr<Memory::MemoryViewWrite> &memoryView) = 0;
virtual bool StartWrite(AuUInt64 offset, const AuSPtr<Memory::MemoryViewRead> &memoryView) = 0;
virtual bool Complete() = 0;
virtual AuUInt32 GetLastPacketLength() = 0;
virtual void SetCallback(const AuSPtr<IAsyncFinishedSubscriber> &sub) = 0;
virtual bool Wait(AuUInt32 timeout) = 0;
virtual AuSPtr<Loop::ILoopSource> NewLoopSource() = 0;
};
}

59
Include/Aurora/IO/FS/IFileStream.hpp
View File

@ -1,17 +1,64 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IFileStream.hpp
Date: 2022-01-17 (wrong)
Author: Reece
***/
#pragma once
namespace Aurora::IO::FS
{
class IFileStream
struct IFileStream
{
public:
virtual AuUInt64 GetOffset() = 0;
virtual bool SetOffset(AuUInt64 offset) = 0;
virtual AuUInt64 GetLength() = 0;
/**
* @brief Read memoryview from the filestream at an auto-incrementing offset
* @param parameters
* @return
*/
virtual bool Read(const Memory::MemoryViewStreamWrite &parameters) = 0;
/**
* @brief Writes memoryview to the filestream at an auto-incrementing offset
* @param parameters
* @return
*/
virtual bool Write(const Memory::MemoryViewStreamRead &parameters) = 0;
virtual void Close() = 0;
/**
* @brief Returns the length of the file
* @return
*/
virtual AuUInt64 GetLength() = 0;
/**
* @brief Returns the offset of the internal streams
* @return
*/
virtual AuUInt64 GetOffset() = 0;
/**
* @brief Updates the stream pointer
* @param offset
* @return
*/
virtual bool SetOffset(AuUInt64 offset) = 0;
/**
* @brief Flush read/write streams
*/
virtual void Flush() = 0;
/**
* @brief Termiantes/truncates the file at the current offset
*/
virtual void WriteEoS() = 0;
/**
* @brief Close IFileStream resource
*/
virtual void Close() = 0;
};
}

25
Include/Aurora/IO/FS/Resources.hpp
View File

@ -15,15 +15,29 @@ namespace Aurora::IO::FS
AUKN_SYM bool GetSystemDomain(AuString &path);
/**
* @brief Provides an application specific storage path for local application data, isolated for your Aurora application brand info (defer to the init structure)
* @brief Provides an application specific storage path for user-local application data, isolated with respect to your Aurora application brand info (defer to the init structure)
*/
AUKN_SYM bool GetProfileDomain(AuString &path);
AUKN_SYM bool GetSystemResourcePath(const AuString &fileName, AuString &path);
/**
* @brief Get package path
* This could be a read-only path if relevant to the platform
* This may be `/storage/emulated/0/Android/data/` (`Context#getExternalFilesDir(java.lang.String)`)
* `/Applications/<mac bundle>`
* `\\some uwp path`
* ...on platforms with read-only application specific data directories
* @param path
* @return
*/
AUKN_SYM bool GetPackagePath(AuString &path);
/**
* @brief Pulls the application directory as defined by the operating system standard file system hierarchy
* Otherwise, XDG_CONFIG_CONFIG or %appdata%.
* This directory might be sandboxed to the user or application by the operating system.
*/
AUKN_SYM bool GetAppData(AuString &path);
@ -33,7 +47,9 @@ namespace Aurora::IO::FS
AUKN_SYM bool GetUserHome(AuString &path);
/**
* @brief Global application data that requires no special permissions to access
* @brief Global application data that requires no special permissions to access.
* This could be an SD-Card or user profile.
* This could be the root of the user managable aplication directory.
*/
AUKN_SYM bool GetWritableAppdata(AuString &path);
@ -41,4 +57,9 @@ namespace Aurora::IO::FS
* @brief Global application data that requires special permissions to access, usually configured by a system account during installation
*/
AUKN_SYM bool GetRootAppdata(AuString &path);
/**
* @brief Get user installable application directory
*/
AUKN_SYM bool GetUserProgramsFolder(AuString &path);
}

3
Include/Aurora/IO/IO.hpp
View File

@ -11,9 +11,10 @@
#include "IStreamReader.hpp"
#include "IStreamWriter.hpp"
#include "IArbitraryStreamer.hpp" // arbitrary read stream, dunno what you would want to call thousands_sep
#include "ISeekingReader.hpp" // arbitrary read stream, dunno what you would want to call thousands_sep
#include "Buffered/Buffered.hpp"
#include "FS/FS.hpp"
#include "Net/Net.hpp"
#include "Character/Character.hpp"

7
Include/Aurora/IO/IArbitraryStreamer.hpp → Include/Aurora/IO/ISeekingReader.hpp
View File

@ -1,7 +1,7 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IArbitraryStreamer.hpp
File: ISeekingReader.hpp
Date: 2021-6-10
Author: Reece
***/
@ -9,11 +9,10 @@
namespace Aurora::IO
{
class IArbitraryStreamer
struct ISeekingReader
{
public:
virtual EStreamError Open() = 0;
virtual EStreamError ArbitraryRead(AuUInt32 offset, const Memory::MemoryViewStreamWrite &paramters) = 0;
virtual EStreamError ArbitraryRead(AuUInt offset, const Memory::MemoryViewStreamWrite &paramters) = 0;
virtual void Close() = 0;
};
}

44
Include/Aurora/IO/IStreamReader.hpp
View File

@ -9,47 +9,17 @@
namespace Aurora::IO
{
class IStreamReader
// TODO (Reece): Consider publishing the multiple inverted ext hack for Aurora Interfaces for binding this.
// It would be nice to have cs-style extensions on this interface. Maybe I shouldn't double down
// over a double blocking API. Maybe I should focus on the stream pumpers.
struct IStreamReader
{
public:
virtual EStreamError Open() = 0;
virtual EStreamError Read(const Memory::MemoryViewStreamWrite &paramters) = 0;
virtual void Close() = 0;
EStreamError ReadAll(Memory::ByteBuffer &buffer)
{
static const int kBufferSize = 2048;
AuUInt len;
EStreamError ret;
AuUInt8 temp[kBufferSize];
len = kBufferSize;
ret = EStreamError::eErrorEndOfStream;
buffer = Memory::NewResizableBuffer();
while ((ret = Read(Memory::MemoryViewStreamWrite(temp, len))) == EStreamError::eErrorNone)
{
if (len == 0)
{
break;
}
buffer.Write(temp, len);
if (len != kBufferSize)
{
break;
}
}
if (ret == EStreamError::eErrorEndOfStream)
{
return EStreamError::eErrorNone;
}
return ret;
}
inline EStreamError ReadAll(Memory::ByteBuffer &buffer);
};
}
#include "IStreamReader.inl" // ReadAll utility

47
Include/Aurora/IO/IStreamReader.inl Normal file
View File

@ -0,0 +1,47 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IStreamReader.inl
Date: 2022-2-1
Author: Reece
***/
#pragma once
namespace Aurora::IO
{
EStreamError IStreamReader::ReadAll(Memory::ByteBuffer &buffer)
{
static const int kBufferSize = 2048;
AuUInt len;
EStreamError ret;
AuUInt8 temp[kBufferSize];
len = kBufferSize;
ret = EStreamError::eErrorEndOfStream;
buffer = Memory::NewResizableBuffer();
while ((ret = Read(Memory::MemoryViewStreamWrite(temp, len))) == EStreamError::eErrorNone)
{
if (len == 0)
{
break;
}
buffer.Write(temp, len);
if (len != kBufferSize)
{
break;
}
}
if (ret == EStreamError::eErrorEndOfStream)
{
return EStreamError::eErrorNone;
}
return ret;
}
}

3
Include/Aurora/IO/IStreamWriter.hpp
View File

@ -9,9 +9,8 @@
namespace Aurora::IO
{
class IStreamWriter
struct IStreamWriter
{
public:
virtual EStreamError Open() = 0;
virtual EStreamError Write(const Memory::MemoryViewStreamRead &parameters) = 0;
virtual void Flush() = 0;

80
Include/Aurora/IO/Net/Net.hpp
View File

@ -14,11 +14,11 @@ namespace Aurora::Async
namespace Aurora::IO::Net
{
static const AuUInt16 kMagicPortAny = 65535;
static const AuUInt16 kMagicPortAny = 0;
struct INetworkStream;
struct IBasicSocket;
struct IClientSocket;
struct ISocket;
struct IServer;
AUE_DEFINE(ETransportProtocol, (
@ -60,11 +60,11 @@ namespace Aurora::IO::Net
if (cmp.ip == EIPProtocol::eIPProtocolV4)
{
return memcmp(cmp.v4, this->v4, sizeof(this->v4)) == 0;
return AuMemcmp(cmp.v4, this->v4, sizeof(this->v4)) == 0;
}
else
{
return memcmp(cmp.v6, this->v6, sizeof(this->v6)) == 0;
return AuMemcmp(cmp.v6, this->v6, sizeof(this->v6)) == 0;
}
}
};
@ -104,11 +104,13 @@ namespace Aurora::IO::Net
{
IPAddress ip;
AuUInt16 port;
AuUInt8 hint[32] {0};
};
struct ConnectionEndpoint
{
ETransportProtocol protocol;
IPEndpoint ip;
bool tls {};
bool compressed {};
// 0 - destination is a stateless datagram server
@ -138,18 +140,18 @@ namespace Aurora::IO::Net
AUKN_INTERFACE(IClientSubscriber,
//
AUI_METHOD(void, OnServerConnectSuccess, (const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(void, OnServerConnectFailed, (const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(void, OnServerConnectSuccess, (const AuSPtr<ISocket> &, socket)),
AUI_METHOD(void, OnServerConnectFailed, (const AuSPtr<ISocket> &, socket)),
// DTLS/UDP/TCP/TLS -> TRUE = expects another datagram or read pump
// FALSE = end of socket life
AUI_METHOD(bool, OnSockeData, (const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(bool, OnSocketData, (const AuSPtr<ISocket> &, socket)),
//
AUI_METHOD(void, OnSocketError, (const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(void, OnSocketError, (const AuSPtr<ISocket> &, socket)),
//
AUI_METHOD(void, OnSocketShutdown, (const AuSPtr<IClientSocket> &, socket))
AUI_METHOD(void, OnSocketShutdown, (const AuSPtr<ISocket> &, socket))
);
AUKN_INTERFACE(IClientSubscriberTls,
@ -158,16 +160,16 @@ namespace Aurora::IO::Net
);
AUKN_INTERFACE(IServerSubscriber,
AUI_METHOD(bool, OnClientAccept, (const AuSPtr<IServer> &, server, const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(bool, OnClientDoS, (const AuSPtr<IServer> &, server, const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(void, OnClientError, (const AuSPtr<IServer> &, server, const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(void, OnClientShutdown, (const AuSPtr<IServer> &, server, const AuSPtr<IClientSocket> &, socket)),
AUI_METHOD(bool, OnReadFrame, (const AuSPtr<IServer> &, server, const AuList<AuSPtr<IClientSocket>> &, sockets)),
AUI_METHOD(bool, OnClientAccept, (const AuSPtr<IServer> &, server, const AuSPtr<ISocket> &, socket)),
AUI_METHOD(bool, OnClientDoS, (const AuSPtr<IServer> &, server, const AuSPtr<ISocket> &, socket)),
AUI_METHOD(void, OnClientError, (const AuSPtr<IServer> &, server, const AuSPtr<ISocket> &, socket)),
AUI_METHOD(void, OnClientShutdown, (const AuSPtr<IServer> &, server, const AuSPtr<ISocket> &, socket)),
AUI_METHOD(bool, OnReadFrame, (const AuSPtr<IServer> &, server, const AuList<AuSPtr<ISocket>> &, sockets)),
AUI_METHOD(void, OnShutdown, (const AuSPtr<IServer> &, server))
);
AUKN_INTERFACE(IServerSubscriberTls,
AUI_METHOD(bool, OnClientTLSReport, (const AuSPtr<IServer> &, server, const AuSPtr<IClientSocket> &, socket, const TLSHandshakeError &, error))
AUI_METHOD(bool, OnClientTLSReport, (const AuSPtr<IServer> &, server, const AuSPtr<ISocket> &, socket, const TLSHandshakeError &, error))
);
// TODO: We should introduce another std:: customer overloadable type reproducing hardcoded ascii and an int, basically std::error_code
@ -235,19 +237,8 @@ namespace Aurora::IO::Net
AuUInt32 bufferedWriteSize {4096 * 10}; // see: enableBufferedOutput
};
struct IClientSocket : public IBasicSocket
struct ISocketChannel
{
virtual bool GetRemoteEndpoint(ConnectionEndpoint &out) = 0;
virtual bool GetLocalEndpoint(ConnectionEndpoint &out) = 0;
virtual bool PumpRead() = 0;
virtual bool PumpWrite() = 0;
virtual bool Pump() = 0;
virtual void Run(int idx, AuUInt32 timeout) = 0;
// If memory.ptr is a nullptr, this method immediately returns with the expected write length in memory.out
//
// If all is true and the internal buffer is not saturated enough yet, no data is read and
@ -333,7 +324,21 @@ namespace Aurora::IO::Net
virtual void ReconfigureStreams(const StreamConfig &config) = 0;
};
struct ILocalClientSocket : public IClientSocket
struct IBasicSocketThreaded : public IBasicSocket
{
virtual bool PumpRead() = 0;
virtual bool PumpWrite() = 0;
virtual bool Pump() = 0;
virtual void Run(int idx, AuUInt32 timeout) = 0;
};
struct ISocket : public IBasicSocketThreaded, public ISocketChannel
{
virtual bool GetRemoteEndpoint(ConnectionEndpoint &out) = 0;
};
struct ILocalClientSocket : public ISocket
{
// Connects to the endpoint defined in the ClientConfig
// Completion will be notified by the following callbacks;
@ -389,9 +394,10 @@ namespace Aurora::IO::Net
AuSPtr<IClientSubscriberTls> clientSubscriber;
};
struct IServer : public IBasicSocket
struct IServer : public IBasicSocketThreaded
{
virtual void GetClients(AuList<AuSPtr<IClientSocket>> &clients) = 0;
virtual bool GetLocalEndpoint(ConnectionEndpoint &out) = 0;
virtual void GetClients(AuList<AuSPtr<IBasicSocketThreaded>> &clients) = 0;
virtual bool Listen() = 0;
virtual void ReconfigureDefaultStream(const StreamConfig &config) = 0;
};
@ -414,10 +420,11 @@ namespace Aurora::IO::Net
struct INetworkInterface
{
virtual IPEndpoint ResolveSocketSync(const SocketHostName &hostname, AuUInt16 port);
virtual IPEndpoint ResolveServiceSync(const ServiceEndpoint &service);
virtual AuList<IPEndpoint> ResolveSocketSync(const SocketHostName &hostname, AuUInt16 port) = 0;
virtual AuList<IPEndpoint> ResolveServiceSync(const ServiceEndpoint &service) = 0;
virtual bool SendDatagramAsync(const ConnectionEndpoint &endpoint, const Memory::MemoryViewRead &memory);
virtual bool SendDatagramAsync(const ConnectionEndpoint &endpoint, const Memory::MemoryViewRead &memory) = 0;
virtual bool SendDatagramAsync(const AuSPtr<IServer> &datagramServer, const ConnectionEndpoint &endpoint, const Memory::MemoryViewRead &memory) = 0;
virtual AuSPtr<ISocketFactory> GetSocketFactory() = 0;
};
@ -453,12 +460,14 @@ namespace Aurora::IO::Net
virtual AuUInt32 PollWorker(AuUInt8 workerId) = 0;
virtual AuUInt32 RunWorker(AuUInt8 workerId, AuUInt32 timeout) = 0;
// D: class APIs
// D:
virtual bool BeginReadPollingOnWorkQueues(const WorkPoolGroup &workGroup) = 0;
virtual bool BeginSubmissionsOnOnWorkQueues(const WorkPoolGroup &workGroup) = 0;
virtual void StopPollingOnWorkQueues() = 0;
// E:
// ..
virtual AuUInt8 GetWorkers() = 0;
virtual AuSPtr<INetworkInterface> GetNetworkInterface() = 0;
@ -466,7 +475,6 @@ namespace Aurora::IO::Net
virtual void Shutdown() = 0;
};
struct NetworkPool
{
AuUInt8 workers {1};

26
Include/Aurora/Locale/ECodePage.hpp
View File

@ -9,21 +9,35 @@
namespace Aurora::Locale
{
enum class ECodePage
{
eUnsupported,
AUE_DEFINE(ECodePage, (
eUTF32,
eUTF32BE,
eUTF16,
eUTF16BE,
// Extended 6 byte UTF8, capable of encoding disallowed characters in recent specs
eUTF8,
// Barely supported
eUTF7,
// GB2312 -> (Microsofts non-standard) GBK <-> GB 18030
// GBK is the relevant subset of GB 18030, implemented by MS and iconv
// GB2312 is not so relevant
// GB 18030 is a backwards compatible modern spec
e18030,
e2312,
eGBK,
// SHIFT_JIS
eSJIS,
// Latin-x
eLatin1,
eSysUnk,
eMax = eUnsupported
};
// System Codepage
eSysUnk
));
}

14
Include/Aurora/Locale/Encoding/Encoding.hpp
View File

@ -19,11 +19,12 @@ namespace Aurora::Locale::Encoding
AUKN_SYM BOM DecodeBOM(const Memory::MemoryViewRead &binary);
// General purpose arbitrary page to UTF8 (AuStrings are UTF-8 - not 16 or 32; bite me)
AUKN_SYM AuStreamReadWrittenPair_t EncodeUTF8(const Memory::MemoryViewRead &utf8, const Memory::MemoryViewWrite &binary, ECodePage page = ECodePage::eUnsupported);
AUKN_SYM AuStreamReadWrittenPair_t DecodeUTF8(const Memory::MemoryViewRead &binary, const Memory::MemoryViewWrite &utf8, ECodePage page = ECodePage::eUnsupported);
AUKN_SYM AuStreamReadWrittenPair_t DecodeUTF8(const Memory::MemoryViewRead &binary, AuString &out, ECodePage page = ECodePage::eUnsupported);
AUKN_SYM AuStreamReadWrittenPair_t EncodeUTF8(const Memory::MemoryViewRead &utf8, const Memory::MemoryViewWrite &binary, ECodePage page = ECodePage::eEnumInvalid);
AUKN_SYM AuStreamReadWrittenPair_t DecodeUTF8(const Memory::MemoryViewRead &binary, const Memory::MemoryViewWrite &utf8, ECodePage page = ECodePage::eEnumInvalid);
AUKN_SYM AuStreamReadWrittenPair_t DecodeUTF8(const Memory::MemoryViewRead &binary, AuString &out, ECodePage page = ECodePage::eEnumInvalid);
// Optimized UTF translation functions
// Note: these functions support full 6-byte UTF8 encoding. 5&6 are reserved.
AUKN_SYM AuStreamReadWrittenPair_t ReadUTF32IntoUTF8ByteString(const Memory::MemoryViewRead &utf32, const Memory::MemoryViewWrite &utf8);
AUKN_SYM AuStreamReadWrittenPair_t ReadUTF8IntoUTF32ByteString(const Memory::MemoryViewRead &utf8, const Memory::MemoryViewWrite &utf32);
@ -34,7 +35,10 @@ namespace Aurora::Locale::Encoding
// Counst the amount of codepoints in a buffer, breaking when the stream is incomplete, giving you the accurate amount of bytes or relevant codepoints in a stream view
AUKN_SYM AuUInt32 CountUTF32Length(const Memory::MemoryViewRead &utf32, bool bytes = false); // codepoint = U32 encoded; always 4 bytes per codepoint
AUKN_SYM AuUInt32 CountUTF16Length(const Memory::MemoryViewRead &utf16, bool bytes = false); // codepoint = U32 encoded; at most: 4 bytes per codepoint, usual: 2 bytes
AUKN_SYM AuUInt32 CountUTF8Length(const Memory::MemoryViewRead &utf8, bool bytes = false); // codepoint = U32 encoded; at most: 6 bytes per codepoint
AUKN_SYM AuUInt32 CountSJISLength(const Memory::MemoryViewRead &sjis, bool bytes = false); // codepoint = one character
AUKN_SYM AuUInt32 CountUTF8Length (const Memory::MemoryViewRead &utf8, bool bytes = false); // codepoint = U32 encoded; at most: 6 bytes per codepoint
AUKN_SYM AuUInt32 CountSJISLength (const Memory::MemoryViewRead &sjis, bool bytes = false); // codepoint = one character
AUKN_SYM AuUInt32 CountGBK16Length(const Memory::MemoryViewRead &gbk, bool bytes = false); // codepoint = at most; one GBK byte pair
AUKN_SYM AuUInt32 CountEncodedStringLength(ECodePage page, const Memory::MemoryViewRead &view, bool bytes = false);
}

2
Include/Aurora/Locale/Locale.hpp
View File

@ -20,7 +20,7 @@ namespace Aurora::Locale
const AuString &language; /// ISO 639
const AuString &country; /// ISO 3166
const AuString &codeset; ///
const ECodePage codepage; /// Potentially eSysUnk; noting that eSysUnk is valid and handlable by the internal backend
const ECodePage codepage; /// Potentially eSysUnk. Note that eSysUnk is valid and handlable by iconv and nsl. On windows and unix, this is your codepage.
};
/*

10
Include/Aurora/Locale/LocaleStrings.hpp
View File

@ -1,3 +1,10 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: LocaleStrings.hpp
Date: 2022-1-24
Author: Reece
***/
#pragma once
#include <Aurora/Time/ETimezoneShift.hpp>
@ -10,10 +17,11 @@ namespace Aurora::Locale
AUKN_SYM const AuString &NewLine();
AUKN_SYM const AuString &TimeLocaleGetMSChar();
AUKN_SYM const AuString &TimeLocaleS();
AUKN_SYM const AuString &TimeLocaleGetSChar();
AUKN_SYM const AuString &TimeLocaleGetDayChar();
AUKN_SYM AuString TimeDateToString(const Time::tm &time);
AUKN_SYM AuString TimeDateToFileNameISO8601(const Time::tm &time, Time::ETimezoneShift shift = Time::ETimezoneShift::eUTC);
AUKN_SYM AuString TimeDateToISO8601(const Time::tm &time, Time::ETimezoneShift shift = Time::ETimezoneShift::eUTC);
AUKN_SYM AuString ConvertMSToTimescale(AuUInt32 ms);

25
Include/Aurora/Logging/IBasicSink.hpp Normal file
View File

@ -0,0 +1,25 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IBasicSink.hpp
Date: 2021-11-1
Author: Reece
***/
#pragma once
namespace Aurora::Logging
{
AUKN_INTERFACE(IBasicSink,
// async callback
AUI_METHOD(void, OnMessageBlocking, (AuUInt8, level, const ConsoleMessage &, msg)),
// sync callback
// returns true if async aware, that is, OnMessageBlocking will be scheduled
AUI_METHOD(bool, OnMessageNonblocking, (AuUInt8, level, const ConsoleMessage &, msg)),
// called every configurable MS or during a problematic event warrenting flush
// expect onflush to be called once after a series of OnMessageNonblocking and every N milliseconds (^1)
AUI_METHOD(void, OnFlush, ())
)
}

25
Include/Aurora/Logging/IBasicSinkRB.hpp Normal file
View File

@ -0,0 +1,25 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: IBasicSinkRB.hpp
Date: 2021-11-2
Author: Reece
***/
#pragma once
namespace Aurora::Logging
{
struct IBasicSinkRB : IBasicSink
{
/**
* @brief Writes the log buffer to a file as though it were an FIO sink
*/
virtual void SaveToPath(const AuString &path, bool binary = false) = 0;
/**
* @brief Dumps the ring buffer to an ordered list
* This method does not clear the ring buffer
*/
virtual AuList<ConsoleMessage> Export() = 0;
};
}

21
Include/Aurora/Console/Logging/ILogger.hpp → Include/Aurora/Logging/ILogger.hpp
View File

@ -7,21 +7,24 @@
***/
#pragma once
namespace Aurora::Console::Logging
namespace Aurora::Logging
{
enum class ELogLevel
{
eZero = 0,
eInfo = 0,
using ConsoleMessage = Console::ConsoleMessage;
using EAnsiColor = Console::EAnsiColor;
AUE_DEFINE(ELogLevel,
(
eInfo,
eVerbose,
eError,
eDebug,
eWarn
};
));
static auto const kLogLevelDefault = static_cast<AuUInt8>(kELogLevelMinLegal);
static auto const kLogLevelUsr = static_cast<AuUInt8>(kELogLevelMaxLegal);
static auto const kLogLevelMax = AuUInt8(255);
static auto const kLogLevelDefault = static_cast<AuUInt8>(ELogLevel::eZero);
static auto const kLogLevelUsr = static_cast<AuUInt8>(ELogLevel::eWarn);
static auto const kLogLevelMax = 16;
struct ILogger
{
virtual void WriteMessage(AuUInt8 level, const ConsoleMessage &msg) = 0;

17
Include/Aurora/Console/Logging/Logging.hpp → Include/Aurora/Logging/Logging.hpp
View File

@ -12,8 +12,19 @@
#include "IBasicSinkRB.hpp"
#include "Sinks.hpp"
namespace Aurora::Console::Logging
namespace Aurora::Logging
{
/// Writes a log message to the console subscribers and telemetry sinks
AUKN_SYM void WriteLine(AuUInt8 level, const Console::ConsoleMessage &msg);
/**
* @brief Overloads the ILogger backend of the AuLogXX functions
* @param defaultGlobalLogger
* @return
*/
AUKN_SYM void SetGlobalLogger(const AuSPtr<Logging::ILogger> &defaultGlobalLogger);
#if defined(_AUHAS_FMT)
template<typename ... T>
@ -134,7 +145,7 @@ namespace Aurora::Console::Logging
template<typename ... T> \
static void AuLog ## level(T&& ... args) \
{ \
Aurora::Console::Logging::Log ## level(AuForward<T>(args)...); \
Aurora::Logging::Log ## level(AuForward<T>(args)...); \
}
ADD_AU_GLOBAL_ALIAS(Info)
@ -149,3 +160,5 @@ ADD_AU_GLOBAL_ALIAS(Verbose)
#else
#define AuLogVerboseNoShip(...)
#endif
#undef ADD_AU_GLOBAL_ALIAS

0
Include/Aurora/Console/README.txt → Include/Aurora/Logging/README.txt
View File

61
Include/Aurora/Logging/Sinks.hpp Normal file
View File

@ -0,0 +1,61 @@
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: Sinks.hpp
Date: 2021-11-2
Author: Reece
***/
#pragma once
namespace Aurora
{
struct SocketConsole;
}
namespace Aurora::Logging
{
struct DirectoryLogger
{
AuUInt32 maxLogsOrZero {};
AuUInt32 maxFileSizeOrZero {}; // MB
};
/**
* @brief Constructs a UTF8 output sink.
* Backed by stdconsole; supports posix fd stdin/out localized, visual studio debugger, and conhost
*/
AUKN_SHARED_API(NewStdSink, IBasicSink);
/**
* @brief Unimplemented systemd or service (?) backend
*/
AUKN_SHARED_API(NewOSEventDirectorySink, IBasicSink);
/**
* @brief Constructs a new sink backed by, syslog or eventlog, by name or event source respectively
*/
AUKN_SHARED_API(NewOSNamedEventDirectorySink, IBasicSink, const AuString &name);
/**
* @brief Constructs a text of binary log file sink
*/
AUKN_SHARED_API(NewFileSink, IBasicSink, const AuString &path, bool binary = false);
/**
* @brief Constructs a dedicated log directory subject to erasure as defined by defined DirectoryLogger
*/
AUKN_SHARED_API(NewDirectorySink, IBasicSink, const AuString &path, DirectoryLogger dirInfo, bool binary = false);
// TODO:
AUKN_SHARED_API(NewIPCSink, IBasicSink, const SocketConsole &console);
/**
* @brief Constructs an in-memory ring buffer sink
*/
AUKN_SHARED_API(NewRingLogger, IBasicSinkRB, AuUInt32 maxLogEntries);
/**
* @brief Constructs a logger object from an array of sinks
*/
AUKN_SHARED_API(NewLogger, ILogger, const AuList<AuSPtr<IBasicSink>> &sinks);
}

41
Include/Aurora/Loop/ELoopSource.hpp Normal file
View File

@ -0,0 +1,41 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ELoopSource.hpp
Date: 2022-2-9
Author: Reece
***/
#pragma once
namespace Aurora::Loop
{
enum class ELoopSource
{
eSourceInternalReserved1,
eSourceInternalReserved2,
eSourceInternalReserved3,
eSourceInternalReserved4,
// generic
eSourceSemaphore,
eSourceCV,
eSourceEvent,
eSourceMutex,
eSourceSRW,
eSourceSocketGroup,
eSourceTimer,
eSourceAIO,
eSourceHandle,
// Specific to the runtime subsystem
eSourceAsync,
// glib oses only
eSourceGlib,
// window messge loops
eSourceApple,
eSourceX11,
eSourceWin32
};
}

186
Include/Aurora/Loop/ILoopQueue.hpp Normal file
View File

@ -0,0 +1,186 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ILoopQueue.hpp
Date: 2022-2-12
Author: Reece
***/
#pragma once
namespace Aurora::Loop
{
// This is like a CF RunLoop with the input parameters of NT's WaitMultipleObjects
// This shouldn't be too much heavier than CF's libevent/NT abstraction in the style of a kevent interface with objects
struct ILoopQueue
{
/**
* @brief
* [*] Source add/remove operations should be followed by a Commit() to commit the changes
*
* [*] Calling Source[Add/Remove] during a Wait[Any/All[Ex]] will cause the wait function to
* return false, allowing for the caller to rewait on the updated queue, or to allow
* the thread to account for the object-was-invalidated condition
*
* @param source
* @return
*/
virtual bool SourceAdd(const AuSPtr<ILoopSource> &source) = 0;
/**
* @brief Same behaviour as SourceAdd
* @param source
* @param timeoutMS
* @return
*/
virtual bool SourceAddWithTimeout(const AuSPtr<ILoopSource> &source, AuUInt32 timeoutMS) = 0;
/**
* @brief Remove operations include commit. You do not need to follow a remove with a commit.
* Sources are defacto poped unless subscriber returns false indicating repeated lock attempts are wanted.
* Should no subscriber be registered, the loop source will not be automatically removed
* @param source
*/
virtual bool SourceRemove(const AuSPtr<ILoopSource> &source) = 0;
/**
* @brief Updates the OS watchdog list cache concept after Source[Remove/Add[WithTimeout]]
* @return
*/
virtual bool Commit() = 0;
/**
* @brief
* @return the amount of loop sources added to the queue
*/
virtual AuUInt32 GetSourceCount() = 0;
/**
* @brief Appends a callback handler to be paired with a loop source
*
* Once a loop source become signaled, and a Wait[Any/All] function call is made, loop source subscribers are called
* to filter the final should-remove operation. The final popped objects are returned from Wait[Any[Ex]] and forgotten.
*
* If no ILoopSourceSubscribers are registered, the edge case behaviour is that loop sources will not be automatically
* dequeued. No user code explicitly told us to do anything other than wait, so the default behaviour of Wait[Any[Ex]]
* shall be to not evict loop sources, until at least one callback is registered, or until SourceRemove is called.
*
* Returning true from a ILoopSourceSubscriber will allow the ILoopQueue to evict the loop source
* Returning false from a ILoopSourceSubscriber will prevent eviction, resulting in continued ILoopSourceSubscriber callbacks to all subscribers
*
* @param source
* @param subscriber
* @return
*/
virtual bool AddCallback(const AuSPtr<ILoopSource> &source, const AuSPtr<ILoopSourceSubscriber> &subscriber) = 0;
/**
* @brief Identical behaviour to that of the basic method. In addition, ILoopSourceSubscriberEx::OnTimeout is called
* on violation of the timeout provided to SourceAddWithTimeout
* @param source
* @param subscriber
* @return
*/
virtual bool AddCallbackEx(const AuSPtr<ILoopSource> &source, const AuSPtr<ILoopSourceSubscriberEx> &subscriber) = 0;
/**
* @brief Near identical behaviour of the extended AddCallback method.
* Registers a callback to handle all loop source signaled events.
* @param subscriber
* @return
*/
virtual bool AddCallback(const AuSPtr<ILoopSourceSubscriber> &subscriber) = 0;
/**
* @brief Indicates whether or not all the loop sources are in a signaled state and all
* relevant interfaces, such as internal IPC handlers and ILoopSourceSubscribers,
* have received the update. Merely waiting for all loop sources to finish is not
* sufficient. One must process the signaled states by calling a Wait function
* when the HasAnyPending() condition is valid.
* @return
*/
virtual bool HasFinished() = 0;
/**
* @brief Nonblocking wait-any for all objects in the loop queue
* @return
*/
virtual bool IsSignaled() = 0;
/**
* @brief Waits on all the submitted loop sources until they are all complete or until the timeout has finished.
* Note: the completion of another Wait[All/Any[Ex]] call may result in a
* @param timeout
* @return
*/
virtual bool WaitAll (AuUInt32 timeout = 0) = 0;
/**
* @brief Waits on all the loop sources until at least one is signaled
* @param timeout
* @return
*/
virtual AuUInt32 WaitAny (AuUInt32 timeout = 0) = 0;
/**
* @brief
* @param timeout
* @return
*/
virtual AuList<AuSPtr<ILoopSource>> WaitAnyEx(AuUInt32 timeout = 0) = 0;
/**
* @brief
* Mostly relevant on Windows where the limit is 64 versus horrible BSD-y socket
* optimized interfaces that can handle thousands of fds
*
* In large loop queues, a maximum of
* `sectionTickTime * (sectionDequeCount + clamp(platformSanity, sectionDequeCount))`
* kernel objects can be checked within a given time frame.
*
* What this effectively does is iterate over a subdivided work queue, aligned
* to platformSanity, [i, i+platform sanity] objects can be safely waitany on.
*
* On timeout of sectionTickTime,
* [i, i+sectionDequeCount] of the tick could be checked, however, we know
* [i, i+platform sanity] timeouted so, in the event of a timeout, we can
* asynchronously check [i+platform sanity, ...] to be nicer faster. Since
* we've already spent the maximum allocated time to poll on a mere subset,
* we may as well desperately check sectionDequeCount worth of objects, and
* write-off the time spent blocking. That way, we get upto platform sanity
* of is signaled checks for free in the worst case scenario. The perspective
* that the overshoot is expensive doesn't make sense, when the is signaled
* check is basically free and the timeout on a subset of the req was the
* bottleneck.
*
* Otherwise on success, we know to check in the range of [i, platform sanity
* for at least one alerted object.
*
* Looking anywhere else given any help from a waitany interface would be stupid.
*
* Internal logic will dequeue platform coefficient aligned handles, and the timeout
* optimization shall check [i+platform sanity, i+platform sanity+sectionDequeCount].
* 1 handle =/= 1 loop source, tho usually 1 loop source ~= 1 handle
*
* tl'dr:
* [*] dont use if you cant stall for sectionTickTime if sectionDequeCount of your
* loop sources are unsignaled
* [*] chug paths are intended for observer threads and not for maintaining high
* performance. no sane traditional application would ever have 64+ signalable
* objects stuck in a loop. though if you do, and you dont configure anything,
* the worst case scenario is, WaitAny can take upto a millisecond to discover
* a signaled loop source.
*
* @param sectionTickTime
* @param sectionDequeCount
*/
virtual void ChugPathConfigure(AuUInt32 sectionTickTime, AuSInt sectionDequeCount) = 0;
/**
* @brief Hints that the calling program understands the kernel shouldnt schedule tne entire source list, and instead, we should long poll
*/
virtual void ChugHint(bool value) = 0;
};
AUKN_SYM AuSPtr<ILoopQueue> NewLoopQueue();
}

26
Include/Aurora/Loop/ILoopSource.hpp Normal file
View File

@ -0,0 +1,26 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ILoopSource.hpp
Date: 2022-2-12
Author: Reece
***/
#pragma once
namespace Aurora::Loop
{
struct ILoopSource
{
/**
* @brief Atomic is-signaled-and-latch
* @return
*/
virtual bool IsSignaled() = 0;
/**
* @brief Returns a generic description about the loop source handle
* @return
*/
virtual ELoopSource GetType() = 0;
};
}

36
Include/Aurora/Loop/ILoopSourceSubscriber.hpp Normal file
View File

@ -0,0 +1,36 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ILoopSourceSubscriber.hpp
Date: 2022-2-12
Author: Reece
***/
#pragma once
namespace Aurora::Loop
{
AUKN_INTERFACE(ILoopSourceSubscriber,
/**
* @brief called under Wait[All/Any[Ex]] once an object is signaled
* @return should pop from ILoopQueue
*/
AUI_METHOD(bool, OnFinished, (const AuSPtr<ILoopSource> &, source))
);
AUKN_INTERFACE(ILoopSourceSubscriberEx,
/**
* @brief called under Wait[All/Any[Ex]] once an object is signaled
* @return should pop from ILoopQueue
*/
AUI_METHOD(bool, OnFinished, (const AuSPtr<ILoopSource> &, source)),
/**
* @brief called under Wait[All/Any[Ex]] once an object timesout before being evicted
* @param
* @param AUI_METHOD
* @param AUI_METHOD
*/
AUI_METHOD(void, OnTimeout, (const AuSPtr<ILoopSource> &, source))
);
}

71
Include/Aurora/Loop/Loop.hpp
View File

@ -4,91 +4,63 @@
File: Loop.hpp
Date: 2021-8-21
Author: Reece
Notes: This API class is specifically for kernel objects or similar userland schedular handles, this is not comparable to the async subsystem.
While you may drive low perf user facing apps and small services from this, other services should divide and conquer
* Allow the network subsystem to load balance sockets across a predefined amount of workers,
* Use semaphores instead of massive lengthy arrays of mutexes
* Use grouped long-polling LoopSources to convert kernel or waitable objects to a single time-polled loop source
***/
#pragma once
#include "ELoopSource.hpp"
#include "ILoopSource.hpp"
#include "ILoopSourceSubscriber.hpp"
#include "ILoopQueue.hpp"
namespace Aurora::Loop
{
enum class ELoopSource
{
eSourceInternalReserved1,
eSourceInternalReserved2,
eSourceInternalReserved3,
eSourceInternalReserved4,
// generic
eSourceSemaphore,
eSourceCV,
eSourceEvent,
eSourceMutex,
eSourceSRW,
eSourceTimer,
eSourceAIO,
eSourceHandle,
// Specific to the runtime subsystem
eSourceAsync,
// glib oses only
eSourceGlib,
// window messge loops
eSourceApple,
eSourceX11,
eSourceWin32
};
class ILoopSource
{
public:
virtual bool IsSignaled() = 0;
virtual ELoopSource GetType() = 0;
};
/// @deprecated
AUKN_SYM AuList<AuSPtr<ILoopSource>> WaitMultipleOrObjects(const AuList<AuSPtr<ILoopSource>> &objects, AuUInt32 timeout);
class ILSSemaphore : public ILoopSource
struct ILSSemaphore : public ILoopSource
{
public:
virtual bool AddOne() = 0;
};
class ILSEvent : public ILoopSource
struct ILSEvent : public ILoopSource
{
public:
virtual bool Set() = 0;
virtual bool Reset() = 0;
};
class ILSMutex : public ILoopSource
struct ILSMutex : public ILoopSource
{
public:
virtual bool Unlock() = 0;
};
class IConditionVar : public ILoopSource
struct IConditionVar : public ILoopSource
{
public:
virtual bool Signal() = 0;
virtual bool Broadcast() = 0;
};
class ITimer : public ILoopSource
struct ITimer : public ILoopSource
{
public:
virtual void UpdateTime(AuUInt64 absTimeMs) = 0;
virtual void Stop() = 0;
};
AUKN_SYM AuSPtr<IConditionVar> NewLSCondVar(const AuSPtr<Threading::IWaitable> &primitive);
AUKN_SYM AuSPtr<IConditionVar> NewLSCondVar(const AuSPtr<ILSMutex> &source);
AUKN_SYM AuSPtr<ITimer> NewLSTimer(AuUInt64 absTimeMs);
AUKN_SYM AuSPtr<ITimer> NewLSTimer(AuUInt64 absTimeMs, AuUInt32 reschedMs = 0);
AUKN_SYM AuSPtr<ILSMutex> NewLSMutex();
AUKN_SYM AuSPtr<ILSEvent> NewLSEvent(bool triggerd = false, bool atomicRelease = true, bool permitMultipleTriggers = false);
AUKN_SYM AuSPtr<ILSSemaphore> NewLSSemaphore(AuUInt32 initialCount = 0);
AUKN_SYM AuSPtr<ILoopSource> NewLSOSHandle(AuUInt);
AUKN_SYM AuSPtr<ILoopSource> NewLSAsync();
AUKN_SYM AuSPtr<ILoopSource> NewLSWin32Source();
AUKN_SYM AuSPtr<ILoopSource> NewLSAsync(Async::WorkerPId_t workerPid);
AUKN_SYM AuSPtr<ILoopSource> NewLSFile(const AuSPtr<IO::FS::IAsyncTransaction> &fileTransaction);
AUKN_SYM AuSPtr<ILoopSource> NewLSWin32Source(bool dispatchMessages);
AUKN_SYM AuSPtr<ILoopSource> NewLSAppleSource();
#if defined(X_PROTOCOL)
@ -114,3 +86,4 @@ namespace Aurora::Loop
}
#endif
}

122
Include/Aurora/Memory/ByteBuffer.hpp
View File

@ -7,33 +7,45 @@
***/
#pragma once
#include <Aurora/Locale/ECodePage.hpp>
namespace Aurora::Memory
{
enum class EStringType
{
eStringTerminated,
eStringByte,
eStringWord,
eStringDword,
eStringQword
};
static const auto kBufferPageSize = 512;
//static const auto kBufferBasePower = 8;
static const auto kBufferInitialPower = 9;// -kBufferBasePower; // 4-bit integer
/***
* A bytebuffer object represents a linear, partially-linear resizable, buffer **or** a ring buffer.
* A bytebuffer object represents a exponentially page-scale resizable buffer **or** a ring buffer.
* Trivial, linear, use-once serialization use cases will likely follow the linear fast paths, not that of a
* ring buffer.
*
* Use cases for a ring buffer include wrapping streams for a use case in which the consumer may
* expect arbitrary stream seeks of an otherwise limited consume-once stream
*
* IE;
* EG:
* -> Peeking a header in a datagram, or tcp stream; where instead of freeing the datagram or double
* buffering the network stack when required, a ring buffer is used to prevent reallocation on each frame
* -> Peeking, or seeking back after, compression read. A compression api could be fed on-Sdemand or ad hoc,
* writing to its write head pointer, while never running out of space so long as the decompressed ring
* read head continues moving
*
* Small, linear, write/read-once [de]serialization use cases may elect to allocate a buffer and
* follow the linear fast paths; perhaps even enabling flagExpandable for hopefully-smarter-than-stdvec-scaling
* -> Peeking, or seeking back after, compression read. A compression api could be fed on-known-input-available
* or on-demand, writing to its write head pointer, while never running out of space so long as the
* decompressed ring read head continues moving
*
* Writing:
* Ring buffers scale from the write head, to the read head, potentially going-around in the process
*
* Linear flagExpandable buffers scale from [0, length]; reallocating at end of buffer if flagExpandable is enabled
* Linear flagExpandable buffers scale from [0, length]
* if expanding is enabled,
* realloc(max(size + offset, (offset / kBufferPageSize + 1) * kBufferPageSize))
* ~~realloc(max(size + offset, (offset / kBufferPageSize + [1 or 2]) * kBufferPageSize))~~ wrong, this was too slow for large streams, high write count cases
*
* Deprecates INetworkStream, fixes allocation issues around compression backends
* Superseeds abuse of AuList<AuUInt8> for binary blobs, alongside Memory::Array
@ -72,6 +84,10 @@ namespace Aurora::Memory
AuUInt8 scaleSize;// : 4; screw it.... we should just take 6 * (4/8) up to 32/64, we wont go up a slab allocation bucket, whatever you want to call it
///////////////////////////////////////////////////////////////////////
/**
* @brief Move constructor
* @param buffer
*/
ByteBuffer(ByteBuffer &&buffer)
{
this->base = buffer.base;
@ -92,6 +108,11 @@ namespace Aurora::Memory
buffer.scaleSize = {};
}
/**
* @brief Copy with possible preserve pointers
* @param buffer
* @param preservePointers
*/
ByteBuffer(const ByteBuffer &buffer, bool preservePointers = true)
{
this->base = FAlloc<AuUInt8 *>(buffer.length);
@ -118,6 +139,13 @@ namespace Aurora::Memory
this->scaleSize = buffer.scaleSize;
}
/**
* @brief Copy pointer range into a new ring or byte buffer
* @param in
* @param length
* @param circular
* @param expandable
*/
ByteBuffer(const void *in, AuUInt length, bool circular = false, bool expandable = false) : flagCircular(circular), flagExpandable(expandable), flagReadError(0), flagWriteError(0)
{
this->base = FAlloc<AuUInt8 *>(length);
@ -183,11 +211,14 @@ namespace Aurora::Memory
this->length = length;
this->allocSize = length;
this->readPtr = this->base;
this->writePtr = this->base;
this->writePtr = this->base + length;
this->scaleSize = kBufferInitialPower;
AuMemcpy(this->base, base, length);
}
/**
* @brief Default constructor, allocates an auto-expanding linear bytebuffer
*/
ByteBuffer() : flagCircular(0), flagExpandable(true), flagReadError(0), flagWriteError(0)
{
this->base = {};
@ -214,7 +245,7 @@ namespace Aurora::Memory
this->writePtr = base;
}
// Iterator
// utils: Iterator
inline auline AuUInt8 * data() const;
inline auline AuUInt size() const;
inline auline AuUInt8 * begin() const;
@ -224,29 +255,30 @@ namespace Aurora::Memory
inline void resize(AuUInt size);
inline void reserve(AuUInt size);
// Utils To alternative types
// utils: Utils to alternative types
inline auline AuList<AuUInt8> ToVector() const;
inline AuUInt32 GetAllocationPower() const;
inline operator AuList<AuUInt8>() const;
inline operator MemoryViewRead() const;
// Internal buffer comparison
// utils: Internal buffer comparison
inline bool operator ==(const AuList<AuUInt8> &) const;
inline bool operator ==(const MemoryViewRead &) const;
inline bool operator ==(const ByteBuffer &) const;
// Move assignment
// utils: Move assignment
inline ByteBuffer &operator =(ByteBuffer &&);
// &byteArray[n]
// utils: &byteArray[n]
inline AuUInt8 &operator [](AuUInt idx) const;
// if (byteArray) -> if (byteArray->IsValid())
// utils: if (byteArray) -> if (byteArray->IsValid())
inline operator bool() const;
inline AuList<AuUInt8> RemainingBytesToVector(bool endAtWrite = true) const;
// ... utils are mostly const functions that provide language intrinsics, access, and container-like compatibility
// ByteBuffer specific utils can be found under Utilities
// Seek / Position
inline auline bool ReaderTryGoForward(AuUInt32 offset);
@ -261,6 +293,7 @@ namespace Aurora::Memory
inline auline AuUInt GetReadOffset() const;
inline auline AuUInt GetWriteOffset() const;
inline auline void ResetReadPointer();
inline AuOptional<AuUInt8 *> WriterTryGetWriteHeadFor(AuUInt32 nBytes);
@ -270,13 +303,53 @@ namespace Aurora::Memory
inline auline bool SetBuffer(const void *in, AuUInt length);
inline auline bool SetBuffer(const AuList<AuUInt8> &buffer);
/**
* @brief Releases excess memory (like, shrink to fit in c++)
* @return
*/
inline auline void GC();
/**
* @brief Releases all resources and resets the bytebuffer without an allocation
*/
inline void Reset();
/**
* @brief Expands the underlying buffer allocation to at least length.
* Does nothing on failure. Programs can try to allocate in real time
* and handle the write error flag condition from there. To pull the
* real buffer size, see member field allocSize.
* @param length
*/
inline void Reserve(AuUInt length);
/**
* @brief Is allocated or dummy object?
* @return
*/
inline auline bool IsEmpty() const;
/**
* @brief Is an error flag set?
* @return
*/
inline auline bool HasStreamError() const;
/**
* @brief Returns true so long as
* 1) an error has not occured or,
* 1) the relevant error flag was reset; and
* 2) there is a valid underlying buffer
*
* @return
*/
inline auline bool IsValid() const;
/**
* @brief Allocate at least length bytes, without adjusting the relative read/write head offsets
* @param length
* @return
*/
inline auline bool Resize(AuUInt length);
// Basic Read Write
@ -284,6 +357,19 @@ namespace Aurora::Memory
inline auline AuUInt Write(const void *buffer, AuUInt requestLength);
inline auline AuUInt Read(void *out, AuUInt requestedLength, bool peek = false);
// String API
inline bool WriteString(const AuString &string, EStringType type, Locale::ECodePage codepage = Locale::ECodePage::eUTF8);
inline bool ReadString(AuString &string, EStringType type, Locale::ECodePage codepage = Locale::ECodePage::eUTF8);
// Copy, concat, etc
inline bool WriteFrom(ByteBuffer &buffer, AuUInt length);
// Utilities
inline bool Trim(AuUInt tail);
inline bool Pad(AuUInt16 aPowOf2, AuUInt8 magicCharacter = '\x00');
inline bool Fill(AuUInt length, AuUInt8 magicCharacter = '\x00');
// Typed read/write
template<typename T>
T Read();

39
Include/Aurora/Memory/ByteBufferPushReadState.hpp Normal file
View File

@ -0,0 +1,39 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ByteBufferPushReadState.hpp
Date: 2022-2-15
Author: Reece
***/
#pragma once
namespace Aurora::Memory
{
struct ByteBufferPushReadState
{
ByteBuffer &bytebuffer;
ByteBufferPushReadState(ByteBuffer &bytebuffer, bool pushError = false) : bytebuffer(bytebuffer), shouldPopError_(pushError)
{
this->readPos_ = this->bytebuffer.readPtr - this->bytebuffer.base;
this->readErrorFlag_ = bytebuffer.flagReadError;
}
~ByteBufferPushReadState()
{
if (this->readErrorFlag_ != bytebuffer.flagReadError)
{
if (this->shouldPopError_)
{
this->bytebuffer.flagReadError = this->readErrorFlag_;
}
this->bytebuffer.readPtr = this->bytebuffer.base + this->readPos_;
}
}
private:
bool readErrorFlag_;
AuUInt readPos_;
const bool shouldPopError_;
};
}

39
Include/Aurora/Memory/ByteBufferPushWriteState.hpp Normal file
View File

@ -0,0 +1,39 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ByteBufferPushWriteState.hpp
Date: 2022-2-15
Author: Reece
***/
#pragma once
namespace Aurora::Memory
{
struct ByteBufferPushWriteState
{
ByteBuffer &bytebuffer;
ByteBufferPushWriteState(ByteBuffer &bytebuffer, bool pushError = false) : bytebuffer(bytebuffer), shouldPopError_(pushError)
{
this->writePos_ = this->bytebuffer.writePtr - this->bytebuffer.base;
this->writeErrorFlag_ = bytebuffer.flagWriteError;
}
~ByteBufferPushWriteState()
{
if (this->writeErrorFlag_ != bytebuffer.flagWriteError)
{
if (this->shouldPopError_)
{
this->bytebuffer.flagWriteError = this->writeErrorFlag_;
}
this->bytebuffer.writePtr = this->bytebuffer.base + this->writePos_;
}
}
private:
bool writeErrorFlag_;
AuUInt writePos_;
const bool shouldPopError_;
};
}

33
Include/Aurora/Memory/ByteBuffer_Memory.inl
View File

@ -49,6 +49,7 @@ namespace Aurora::Memory
if (this->base)
{
Free(this->base);
this->base = {};
}
this->base = {};
@ -62,16 +63,32 @@ namespace Aurora::Memory
void ByteBuffer::Reserve(AuUInt length)
{
auto oldLength = this->length;
if (length > this->allocSize)
{
this->Resize(length);
this->length = AuMin(oldLength, length);
}
this->length = AuMin(oldLength, this->length);
}
void ByteBuffer::GC()
{
if (this->length)
if (this->allocSize == this->length) return;
auto temp = Memory::FRealloc(this->base, this->length);
if (!temp) return;
if (!this->base)
{
return;
}
if (this->allocSize == this->length)
{
return;
}
auto temp = Memory::ZRealloc(this->base, this->length);
if (!temp)
{
return;
}
this->base = temp;
this->length = this->length;
this->allocSize = this->length;
@ -84,7 +101,9 @@ namespace Aurora::Memory
if (length == 0)
{
Reset();
this->length = 0;
this->writePtr = this->base;
this->readPtr = this->base;
return true;
}
@ -109,7 +128,7 @@ namespace Aurora::Memory
{
auto scale = GetAllocationPower();
oldLength = this->length;
newLength = AuMax(AuUInt(length), AuUInt(((this->allocSize / scale) + 1) * scale));
newLength = AuMax(AuUInt(length), (((this->allocSize * 2) / scale) + 1) * scale);// AuUInt(((this->allocSize / scale) + 2) * scale)); < this spends too much time spinning
nextPtr = ZRealloc(this->base, newLength);
if (!nextPtr)

33
Include/Aurora/Memory/ByteBuffer_Position.inl
View File

@ -11,19 +11,37 @@ namespace Aurora::Memory
{
bool ByteBuffer::WriterTryGoForward(AuUInt32 offset)
{
if (!offset)
{
return true;
}
auto old = writePtr - base;
auto n = Write(nullptr, offset);
SysAssert(n == offset); // TODO: writer go back
if (n != offset)
{
this->writePtr = this->base + old;
return false;
}
return true;
}
bool ByteBuffer::ReaderTryGoForward(AuUInt32 offset)
{
if (!offset)
{
return true;
}
auto old = readPtr - base;
auto n = Read(nullptr, offset, false);
if (n != offset)
{
SysAssert(ReaderTryGoBack(n));
this->readPtr = this->base + old;
return false;
}
return true;
}
@ -70,12 +88,12 @@ namespace Aurora::Memory
if (readOffset < offset)
{
auto absPosition = offset - readOffset;
auto absPosition = (AuUInt)offset - readOffset;
auto goAround = absPosition;
auto backIdx = length - goAround;
auto writeOffset = writePtr - base;
if (writeOffset > backIdx)
if ((AuUInt)writeOffset > (AuUInt)backIdx)
{
return false;
}
@ -118,7 +136,7 @@ namespace Aurora::Memory
{
if ((readPtr < writePtr) && (endAtWrite))
{
return length - (writePtr - readPtr);
return writePtr - readPtr;
}
else
{
@ -180,6 +198,11 @@ namespace Aurora::Memory
return true;
}
void ByteBuffer::ResetReadPointer()
{
readPtr = base;
}
AuUInt ByteBuffer::GetReadOffset() const
{
if (flagCircular)

31
Include/Aurora/Memory/ByteBuffer_ReadWrite.inl
View File

@ -66,11 +66,22 @@ namespace Aurora::Memory
else
{
auto offset = writePtr - base;
if ((AuUInt)offset > (AuUInt)length)
{
SysPushErrorIO();
this->flagWriteError = true;
return 0;
}
auto overhead = length - offset;
AuUInt len = AuMin(overhead, requestLength);
if ((len != requestLength) && (flagExpandable))
if ((len != requestLength))
{
if ((flagExpandable))
{
if (!Resize(offset + requestLength))
{
@ -80,6 +91,11 @@ namespace Aurora::Memory
overhead = length - offset;
len = AuMin(overhead, requestLength);
}
else
{
if (len == 0) return 0;
}
}
if (buffer)
{
@ -87,11 +103,11 @@ namespace Aurora::Memory
}
writePtr += len;
SysAssert(writePtr <= base + length);
return len;
}
}
AuUInt ByteBuffer::Read(void *out, AuUInt requestedLength, bool peek)
{
AuUInt linearOverhead = 0, toWriteOverhead = 0, linearReadable = 0, toWriteReadable = 0;
@ -124,8 +140,11 @@ namespace Aurora::Memory
}
if (toWriteOverhead)
{
if (out)
{
AuMemcpy(reinterpret_cast<AuUInt8 *>(out) + linearOverhead, base, toWriteReadable);
}
if (!peek)
{
@ -144,6 +163,13 @@ namespace Aurora::Memory
}
else
{
if (readPtr > writePtr)
{
SysPushErrorIO();
this->flagReadError = true;
return 0;
}
AuUInt len = AuMin(AuUInt(writePtr - readPtr), requestedLength);
if (out)
@ -159,5 +185,4 @@ namespace Aurora::Memory
return len;
}
}
}

137
Include/Aurora/Memory/ByteBuffer_Strings.inl Normal file
View File

@ -0,0 +1,137 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ByteBuffer_Strings.inl
Date: 2022-2-15
Author: Reece
***/
#pragma once
namespace Aurora::Memory
{
bool ByteBuffer::WriteString(const AuString &string, EStringType type, Locale::ECodePage codepage)
{
ByteBufferPushWriteState a(*this);
AuStreamReadWrittenPair_t len {};
Memory::ByteBuffer temp;
if (codepage != Locale::ECodePage::eUTF8)
{
len = Aurora::Locale::Encoding::EncodeUTF8(string, {}, codepage);
if (len == AuStreamReadWrittenPair_t {0, 0})
{
return false;
}
temp = Memory::ByteBuffer(len.second);
len = Aurora::Locale::Encoding::EncodeUTF8(string, Memory::MemoryViewStreamWrite(temp.begin(), temp.end()), codepage);
if (len == AuStreamReadWrittenPair_t {0, 0})
{
return false;
}
}
else
{
len = AuStreamReadWrittenPair_t {AuUInt32(string.size()), AuUInt32(string.size())};
}
switch (type)
{
case EStringType::eStringByte:
{
if (len.second > 255)
{
return {};
}
if (!Write<AuUInt8>(static_cast<AuUInt8>(len.second)))
{
return {};
}
break;
}
case EStringType::eStringWord:
{
if (len.second > std::numeric_limits<AuUInt16>::max())
{
return {};
}
if (!Write<AuUInt16>(static_cast<AuUInt16>(len.second)))
{
return {};
}
break;
}
case EStringType::eStringDword:
{
if (len.second > std::numeric_limits<AuUInt32>::max())
{
return {};
}
if (!Write<AuUInt32>(static_cast<AuUInt32>(len.second)))
{
return {};
}
break;
}
case EStringType::eStringQword:
{
if (!Write<AuUInt64>(static_cast<AuUInt64>(len.second)))
{
return {};
}
break;
}
}
if (codepage != Locale::ECodePage::eUTF8)
{
if (!Write(temp.base, temp.length))
{
return {};
}
}
else
{
if (!Write(string.data(), string.size()))
{
return {};
}
}
if (type == EStringType::eStringTerminated)
{
if (codepage == Locale::ECodePage::eUTF16 || codepage == Locale::ECodePage::eUTF16BE)
{
if (!Write<AuUInt16>(0))
{
return {};
}
}
else if (codepage == Locale::ECodePage::eUTF32 || codepage == Locale::ECodePage::eUTF32BE)
{
if (!Write<AuUInt32>(0))
{
return {};
}
}
else
{
if (!Write<AuUInt8>(0))
{
return {};
}
}
}
return true;
}
bool ByteBuffer::ReadString(AuString &string, EStringType type, Locale::ECodePage codepage)
{
// TODO: ...
return {};
}
}

79
Include/Aurora/Memory/ByteBuffer_Utility.inl Normal file
View File

@ -0,0 +1,79 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ByteBuffer_Utility.inl
Date: 2022-2-15
Author: Reece
***/
#pragma once
namespace Aurora::Memory
{
bool ByteBuffer::Trim(AuUInt tail)
{
if (this->length > tail)
{
return false;
}
return Resize(this->length - tail);
}
bool ByteBuffer::Fill(AuUInt length, AuUInt8 magicCharacter)
{
ByteBufferPushWriteState a(*this);
auto maxLinear = this->writePtr - AuMin(this->readPtr, this->base + this->length);
maxLinear = AuMin(AuUInt(maxLinear), length);
if (maxLinear)
{
AuMemset(this->writePtr, magicCharacter, maxLinear);
this->writePtr += maxLinear;
}
if (maxLinear != length)
{
auto delta = length - maxLinear;
// HACK:
Memory::ByteBuffer inner(delta);
if (!inner)
{
return {};
}
AuMemset(inner.base, magicCharacter, inner.length);
if (!Write(inner.base, inner.length))
{
return {};
}
}
return true;
}
bool ByteBuffer::Pad(AuUInt16 aPowOf2, AuUInt8 magicCharacter)
{
ByteBufferPushWriteState a(*this);
auto oldLen = this->length;
auto newLen = ((this->length + aPowOf2) & ~(aPowOf2 - 1));
if (!Resize(newLen))
{
return {};
}
auto old = this->writePtr - this->base;
this->writePtr = this->base + oldLen;
if (!this->Fill(newLen - oldLen))
{
return {};
}
this->writePtr = this->base + old;
return true;
}
}

2
Include/Aurora/Memory/ByteBuffer_Utils.inl
View File

@ -41,6 +41,7 @@ namespace Aurora::Memory
AuUInt8 &ByteBuffer::operator [](AuUInt idx) const
{
SysAssert(idx < length);
return *(data() + idx);
}
@ -119,7 +120,6 @@ namespace Aurora::Memory
return !IsEmpty() && !HasStreamError();
}
bool ByteBuffer::IsEmpty() const
{
return !length || !base;

16
Include/Aurora/Memory/ByteBuffer_WriteFrom.inl Normal file
View File

@ -0,0 +1,16 @@
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ByteBuffer_WriteFrom.inl
Date: 2022-2-15
Author: Reece
***/
#pragma once
namespace Aurora::Memory
{
bool ByteBuffer::WriteFrom(ByteBuffer &buffer, AuUInt length)
{
return {};
}
}

12
Include/Aurora/Memory/ExtendStlLikeSharedPtr.hpp
View File

@ -35,25 +35,29 @@ namespace Aurora::Memory
return *this;
}
TType_t &operator*() const
template<typename TType2_t = TType_t>
TType2_t &operator*() const
{
throwif();
return Base_t::operator*();
}
TType_t *operator->() const
template<typename TType2_t = TType_t>
TType2_t *operator->() const
{
throwif();
return Base_t::operator->();
}
TType_t &operator*()
template<typename TType2_t = TType_t>
TType2_t &operator*()
{
throwif();
return Base_t::operator*();
}
TType_t *operator->()
template<typename TType2_t = TType_t>
TType2_t *operator->()
{
throwif();
return Base_t::operator->();

6
Include/Aurora/Memory/Memory.hpp
View File

@ -83,7 +83,7 @@ namespace Aurora::Memory
{
}
[[nodiscard]] T *allocate(std::size_t n)
[[nodiscard]] T *allocate(Types::size_t n)
{
if (auto p = (NewArray<T>(n)))
{
@ -93,7 +93,7 @@ namespace Aurora::Memory
throw std::bad_alloc();
}
void deallocate(T *p, std::size_t n) noexcept
void deallocate(T *p, Types::size_t n) noexcept
{
_Free(p);
}
@ -146,3 +146,5 @@ namespace Aurora::Memory
}
#include "ByteBuffer.hpp"
#include "ByteBufferPushReadState.hpp"
#include "ByteBufferPushWriteState.hpp"

34
Include/Aurora/Memory/MemoryView.hpp
View File

@ -18,11 +18,10 @@ namespace Aurora::Memory
using StdArray_t = AuConditional_t<Readonly_b, const AuArray<T, Z>, AuArray<T, Z>>;
/*
YadaYada(MemoryView(tempstring/array/etc)) should be legal, right?
MethodName(MemoryView(tempstring/array/etc)) should be legal, right?
---------------------
Temporary objects are destroyed as the last step in evaluating the full-expression that (lexically) contains the point where they were created.
---------------------
A full-expression is:
[...]
* an expression that is not a subexpression of another expression and that is not otherwise part of a full-expression.
@ -103,9 +102,9 @@ namespace Aurora::Memory
}
template<typename T>
T *Begin() const
AuConditional_t<Readonly_b, const T *, T *> Begin() const
{
return reinterpret_cast<T *>(ptr);
return reinterpret_cast<AuConditional_t<Readonly_b, const T *, T *>>(ptr);
}
template<typename T>
@ -119,6 +118,11 @@ namespace Aurora::Memory
return ptr && length;
}
operator bool() const
{
return HasMemory();
}
Void_t /*const*/ ptr;
AuUInt /*const*/ length;
};
@ -169,12 +173,32 @@ namespace Aurora::Memory
outVariable = 0;
}
constexpr MemoryViewStream(MemoryView<Readonly_b> in) : MemoryView<Readonly_b>(in.ptr, in.length), outVariable(unused)
{
outVariable = 0;
}
constexpr MemoryViewStream(MemoryView<Readonly_b> in, AuUInt &len) : MemoryView<Readonly_b>(in.ptr, in.length), outVariable(len)
{
outVariable = 0;
}
template<typename T, int Z>
constexpr MemoryViewStream(T(&a)[Z]) : MemoryView<Readonly_b>(a), outVariable(unused)
{
outVariable = 0;
}
bool HasMemory() const
{
return this->ptr && this->length;
}
operator bool() const
{
return HasMemory();
}
AuUInt &outVariable;
private:
AuUInt unused;

3
Include/Aurora/Memory/_ByteBuffer.hpp
View File

@ -5,3 +5,6 @@
#include "ByteBuffer_Memory.inl"
#include "ByteBuffer_Utils.inl"
#include "ByteBuffer_Position.inl"
#include "ByteBuffer_Utility.inl"
#include "ByteBuffer_Strings.inl"
#include "ByteBuffer_WriteFrom.inl"

14
Include/Aurora/Parse/Base64.hpp
View File

@ -11,13 +11,21 @@ namespace Aurora::Parse
{
AUKN_SYM bool Base64Decode(const AuString &in, Memory::ByteBuffer &decoded, bool url = false);
AUKN_SYM bool Base64Encode(const void *buffer, AuMach length, AuString &encoded, bool url = false);
AUKN_SYM bool Base64Encode(const Memory::MemoryViewRead &input, AuString &encoded, bool url = false);
static bool Base64Encode(const AuString &in, AuString &encoded, bool url = false)
{
return Base64Encode(in.data(), in.size(), encoded, url);
return Base64Encode(Memory::MemoryViewRead {in}, encoded, url);
}
static bool Base64Encode(const Memory::ByteBuffer &in, AuString &encoded, bool url = false)
{
return Base64Encode(in.data(), in.size(), encoded, url);
return Base64Encode(Memory::MemoryViewRead {in}, encoded, url);
}
template<typename T, typename Z>
static bool Base64Encode(T begin, Z end, AuString &encoded, bool url = false)
{
return Base64Encode(Memory::MemoryViewRead {begin, end}, encoded, url);
}
}

6
Include/Aurora/Parse/Hex.hpp
View File

@ -9,14 +9,14 @@
namespace Aurora::Parse
{
enum class EHexDump
{
AUE_DEFINE(EHexDump,
(
eString,
eZeroXSpace,
eHexEditor,
eCLiteral,
eJSLiteral
};
));
AUKN_SYM void ByteToHex(AuUInt8 val, char(&hex)[2]);
AUKN_SYM bool HexToInt (const char *hex, AuUInt32 length, AuUInt64 &val);

39
Include/Aurora/Parse/Parser.hpp
View File

@ -8,6 +8,10 @@
#pragma once
#include <Aurora/Data/Data.hpp>
#include <Aurora/IO/Character/ICharacterProvider.hpp>
#include <Aurora/IO/Character/ICharacterProviderEx.hpp>
#include <Aurora/IO/Character/IBufferedCharacterConsumer.hpp>
#include <Aurora/IO/Character/Providers.hpp>
namespace Aurora::Parse
{
@ -112,6 +116,7 @@ namespace Aurora::Parse
ParsedObject result;
};
#if 0
using ConsumeStream_cb = AuFunction<bool(AuUInt8 &)>;
static ConsumeStream_cb StringToConsumable(const AuString &str, AuMach &index)
@ -128,17 +133,15 @@ namespace Aurora::Parse
};
return getc;
}
#endif
struct ParseState
{
ParseState(ConsumeStream_cb &str) : stringstream(str)
ParseState(const AuSPtr<Aurora::IO::Character::ICharacterProvider> &stream) : stream(stream)
{}
ParseState(ConsumeStream_cb &&str) : stringstream(AuMove(str))
{}
ConsumeStream_cb stringstream;
AuSPtr<Aurora::IO::Character::ICharacterProvider> stream;
AuUInt8 *additionalTokens {};
AuUInt16 countOfTokens {};
@ -149,31 +152,41 @@ namespace Aurora::Parse
};
AUKN_SYM void VaildateStructure(const ParseObject &object);
AUKN_SYM bool ConsumeToken(ParsableTag type, ConsumeStream_cb getc, ParseValue &out);
AUKN_SYM bool ConsumeToken(ParsableTag type, const AuSPtr<Aurora::IO::Character::ICharacterProvider> &getc, ParseValue &out);
static bool ConsumeToken(ParsableTag type, const AuString &str, AuMach &index, ParseValueEx &out)
{
return ConsumeToken(type, StringToConsumable(str, index), out);
auto strStream = IO::Character::ProviderFromStringUnique(str, index);
if (!strStream) return false;
if (!ConsumeToken(type, AuUnsafeRaiiToShared(strStream), out)) return false;
index = strStream->GetPosition();
return true;
}
static bool ConsumeToken(ParsableTag type, const AuString &str, ParseValueEx &out)
{
AuMach index{};
return ConsumeToken(type, StringToConsumable(str, index), out);
auto strStream = IO::Character::ProviderFromStringUnique(str);
if (!strStream) return false;
return ConsumeToken(type, AuUnsafeRaiiToShared(strStream), out);
}
AUKN_SYM bool Parse(ParseState &state, const ParseObject &structure, ParseResult &result);
static bool Parse(ParseResult &result, const ParseObject &structure, const AuString &str, AuMach &index)
{
ParseState state(StringToConsumable(str, index));
return Parse(state, structure, result);
auto strStream = IO::Character::ProviderFromStringUnique(str, index);
if (!strStream) return false;
ParseState state(AuUnsafeRaiiToShared(strStream));
if (!Parse(state, structure, result)) return false;
index = strStream->GetPosition();
return true;
}
static bool Parse(ParseResult &result, const ParseObject &structure, const AuString &str)
{
AuMach index{};
ParseState state(StringToConsumable(str, index));
auto strStream = IO::Character::ProviderFromStringUnique(str, 0);
if (!strStream) return false;
ParseState state(AuUnsafeRaiiToShared(strStream));
return Parse(state, structure, result);
}

Some files were not shown because too many files have changed in this diff Show More