AuroraRuntime/Source/Console/ConsoleStd/ConsoleStd.cpp

1772 lines
53 KiB
C++
Raw Normal View History

2021-09-06 10:58:08 +00:00
/***
Copyright (C) 2021 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ConsoleStd.cpp
Date: 2021-6-8
Author: Reece
***/
2021-09-30 14:57:41 +00:00
#include <Source/RuntimeInternal.hpp>
2021-09-06 10:58:08 +00:00
#include "ConsoleStd.hpp"
#include <Source/Locale/Locale.hpp>
#include <Source/Console/Console.hpp>
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
#include <Source/Logging/Sinks/StdConsole.hpp>
2021-09-06 10:58:08 +00:00
#include <Source/IO/Loop/ILoopSourceEx.hpp>
#include <Source/IO/Loop/LSHandle.hpp>
#include <Source/IO/Loop/LSEvent.hpp>
#include <Source/Console/ConsoleTTY/ConsoleTTY.hpp>
2022-12-14 01:35:18 +00:00
#include "../ColorConvert.hpp"
2021-09-06 10:58:08 +00:00
#if defined(AURORA_IS_MODERNNT_DERIVED) || defined(AURORA_IS_POSIX_DERIVED)
#if defined(AURORA_IS_MODERNNT_DERIVED)
// nothing yet
#elif defined(AURORA_IS_POSIX_DERIVED)
#define IO_POSIX_STREAMS
#include <stdio.h>
#include <sys/select.h>
#include <unistd.h>
#include <termios.h>
2021-09-06 10:58:08 +00:00
#endif
// compile time preprocessor definition to strip std stream based IO
#if !(defined(AURORA_DISABLE_STD_CONSOLE))
#define ENABLE_STD_CONSOLE
#endif
#endif
namespace Aurora::Console::ConsoleStd
{
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
#if defined(ENABLE_STD_CONSOLE)
struct ConsoleHasDataLoopSource :
2023-12-18 01:42:50 +00:00
#if defined(AURORA_IS_MODERNNT_DERIVED)
virtual AuLoop::ILoopSource,
2023-12-18 01:42:50 +00:00
#endif
virtual AuLoop::LSEvent
{
2023-12-18 01:42:50 +00:00
#if defined(AURORA_IS_MODERNNT_DERIVED)
ConsoleHasDataLoopSource() : LSEvent(false, false, true)
{
}
2023-12-18 01:42:50 +00:00
#else
ConsoleHasDataLoopSource(int fd) : LSEvent(false, false, true)
{
2023-12-18 01:42:50 +00:00
handles = { AuUInt(fd), LSEvent::GetHandle() };
}
2023-12-18 01:42:50 +00:00
#endif
bool IsSignaled() override;
AuLoop::ELoopSource GetType() override;
bool OnTrigger(AuUInt handle) override;
bool WaitOn(AuUInt32 timeout) override;
private:
2023-12-18 01:42:50 +00:00
#if !defined(AURORA_IS_MODERNNT_DERIVED)
AuList<AuUInt> handles;
2023-12-18 01:42:50 +00:00
virtual const AuList<AuUInt>& GetHandles() override;
virtual bool Singular() override;
#endif
};
struct ConsoleSecondaryDataLoopSource :
virtual AuLoop::ILoopSource,
virtual AuLoop::LSEvent
{
#if defined(AURORA_IS_MODERNNT_DERIVED)
ConsoleSecondaryDataLoopSource(LSEvent *pAltHandle) : LSEvent(false, true, true)
{
#if defined(STD_INPUT_HANDLE)
handles = { /*pAltHandle->GetHandle()*/AuUInt(GetStdHandle(STD_INPUT_HANDLE)), LSEvent::GetHandle() };
#else
handles = { LSEvent::GetHandle() };
#endif
2023-12-18 01:42:50 +00:00
}
#else
ConsoleSecondaryDataLoopSource(int fd) : LSEvent(false, true, true)
{
handles = { AuUInt(fd), LSEvent::GetHandle() };
}
#endif
bool IsSignaled() override;
AuLoop::ELoopSource GetType() override;
bool OnTrigger(AuUInt handle) override;
bool WaitOn(AuUInt32 timeout) override;
private:
AuList<AuUInt> handles;
virtual const AuList<AuUInt>& GetHandles() override;
virtual bool Singular() override;
};
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
static AuByteBuffer gStdoutBuffer;
2021-09-06 10:58:08 +00:00
#if defined(AURORA_IS_MODERNNT_DERIVED)
#define DEFAULT_HANDLE_VAL INVALID_HANDLE_VALUE
using StreamHandle_t = HANDLE;
2021-09-06 10:58:08 +00:00
static StreamHandle_t gWin32Thread = INVALID_HANDLE_VALUE;
static DWORD gCanonicalBackup {};
2021-09-06 10:58:08 +00:00
#elif defined(IO_POSIX_STREAMS)
#define DEFAULT_HANDLE_VAL 0xFFFFFFFF
2021-09-06 10:58:08 +00:00
using StreamHandle_t = int;
#endif
#define IS_STREAM_HANDLE_VALID(h) (h != DEFAULT_HANDLE_VAL)
2021-09-06 10:58:08 +00:00
static bool AsyncReadAnyOrReadStreamBlock();
void WriteStdOut(AuUInt8 level, const ConsoleMessage &msg, const AuString &formatted);
2021-09-06 10:58:08 +00:00
static const AuMach kLineBufferMax = 2048;
static AuUInt8 gLineEncodedBuffer[kLineBufferMax];
static AuUInt gEncodedIndex = 0;
static AuString gLineBuffer(kLineBufferMax, 0);
static AuUInt gLineIndex = 0;
static StreamHandle_t gTerminateConsole;
static StreamHandle_t gInputStream = DEFAULT_HANDLE_VAL;
static StreamHandle_t gOutputStream = DEFAULT_HANDLE_VAL;
static AuSPtr<ConsoleHasDataLoopSource> gLoopSource;
2023-12-18 01:42:50 +00:00
static AuSPtr<ConsoleSecondaryDataLoopSource> gLoopSourceConTTY;
static bool gCanonicalEnabled {};
static AuList<NoncanonicalInput> gCanonicalBuffer;
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
//static AuThreadPrimitives::MutexUnique_t gRingLock = AuThreadPrimitives::MutexUnique();
static AuMutex gRingLock;// = AuThreadPrimitives::MutexUnique();
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
static bool gBufferMode {};
2021-09-06 10:58:08 +00:00
static bool gCanonicalUnixOn {};
2021-09-06 10:58:08 +00:00
#if defined(AURORA_IS_MODERNNT_DERIVED)
}
#include "../ConsoleTTY/ConsoleTTY.hpp"
#include "../ConsoleTTY/ConsoleTTY.NT.hpp"
namespace Aurora::Console::ConsoleStd
{
2021-09-06 10:58:08 +00:00
static DWORD WINAPI StdInWin32Thread(void*)
{
HANDLE a[2] = {gInputStream, gTerminateConsole};
while (!gCanonicalEnabled)
2021-09-06 10:58:08 +00:00
{
WaitForMultipleObjectsEx(2, a, false, 25, 0);
if (WaitForSingleObject(gTerminateConsole, 0) == WAIT_OBJECT_0)
{
break;
}
AsyncReadAnyOrReadStreamBlock();
}
return 1;
}
static void PumpClipboardCheck()
{
static AuUInt64 gLastHash {};
AuString str;
if (!pOpenClipboard ||
!pCloseClipboard ||
!pGetClipboardData)
{
return;
}
if (!pOpenClipboard(nullptr))
{
return;
}
auto hClipboardData = pGetClipboardData(CF_UNICODETEXT);
if (hClipboardData == nullptr)
{
pCloseClipboard();
return;
}
auto pLpwStr = (LPWSTR)GlobalLock(hClipboardData);
if (pLpwStr != nullptr)
{
str = AuLocale::ConvertFromWChar(pLpwStr);
::GlobalUnlock(hClipboardData);
}
auto newHash = AuFnv1a64Runtime(str.data(), str.length());
if (newHash != gLastHash)
{
if (gLastHash)
{
ConsoleTTY::EnterScrollMode();
}
gLastHash = newHash;
}
pCloseClipboard();
}
static void ProcessLines(AuList<AuString> &lines);
static void TryPasteFromClipboard()
{
AuString str;
if (!pOpenClipboard ||
!pCloseClipboard ||
!pGetClipboardData)
{
return;
}
if (!pOpenClipboard(nullptr))
{
return;
}
auto hClipboardData = pGetClipboardData(CF_UNICODETEXT);
if (hClipboardData == nullptr)
{
pCloseClipboard();
return;
}
auto pLpwStr = (LPWSTR)GlobalLock(hClipboardData);
if (pLpwStr != nullptr)
{
str = AuLocale::ConvertFromWChar(pLpwStr);
::GlobalUnlock(hClipboardData);
}
{
auto remaining = AuParse::SplitNewlines(str, [](const AuROString &line)
{
NoncanonicalInput canInput {};
canInput.type = ENoncanonicalInput::eInput;
canInput.string = line;
if (auto uMax = ::strnlen(line.data(), line.size()))
{
canInput.string.resize(uMax);
}
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
canInput.type = ENoncanonicalInput::eEnter;
canInput.string.clear();
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}, true);
if (remaining.size())
{
NoncanonicalInput canInput {};
canInput.type = ENoncanonicalInput::eInput;
canInput.string = remaining;
if (auto uMax = ::strnlen(remaining.data(), remaining.size()))
{
canInput.string.resize(uMax);
}
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
}
pCloseClipboard();
}
void ProcessCanonical(HANDLE h)
{
#if defined(AURORA_PLATFORM_WIN32)
INPUT_RECORD records[4096];
DWORD dwRecords;
void *data = &gLineEncodedBuffer[gEncodedIndex];
auto length = kLineBufferMax - gEncodedIndex;
PumpClipboardCheck();
if (!IS_STREAM_HANDLE_VALID(h))
{
h = gInputStream;
}
if (length == 0)
{
return;
}
if (!pGetNumberOfConsoleInputEvents ||
!pReadConsoleInputW)
{
return;
}
if (!pGetNumberOfConsoleInputEvents(h,
&dwRecords))
{
return;
}
dwRecords = AuMin(dwRecords, DWORD(AuArraySize(records)));
if (!dwRecords)
{
return;
}
if (!pReadConsoleInputW(h,
records,
dwRecords,
&dwRecords))
{
return;
}
AU_LOCK_GUARD(gRingLock);
gCanonicalBuffer.reserve(4096);
if (dwRecords)
{
if (gLoopSource)
{
gLoopSource->Set();
}
}
for (auto i = 0u; i < dwRecords; i++)
{
int z;
bool dBreak = false;
NoncanonicalInput canInput;
canInput.type = ENoncanonicalInput::eEnumInvalid;
canInput.iScrollDeltaY = 0;
auto &record = records[i];
AuString key;
switch (record.EventType)
{
case KEY_EVENT:
{
if (!record.Event.KeyEvent.bKeyDown)
{
dBreak = true;
break;
}
switch (record.Event.KeyEvent.wVirtualKeyCode)
{
case VK_UP:
canInput.type = ENoncanonicalInput::eArrowUp;
break;
case VK_DOWN:
canInput.type = ENoncanonicalInput::eArrowDown;
break;
case VK_LEFT:
canInput.type = ENoncanonicalInput::eArrowLeft;
break;
case VK_RIGHT:
canInput.type = ENoncanonicalInput::eArrowRight;
break;
case VK_HOME:
canInput.type = ENoncanonicalInput::eHome;
break;
case VK_BACK:
canInput.type = ENoncanonicalInput::eBackspace;
break;
case VK_PRIOR:
canInput.type = ENoncanonicalInput::ePageUp;
break;
case VK_NEXT:
canInput.type = ENoncanonicalInput::ePageDown;
break;
case VK_RETURN:
canInput.type = ENoncanonicalInput::eEnter;
break;
case VK_TAB:
canInput.type = ENoncanonicalInput::eTab;
break;
case VK_DELETE:
canInput.type = ENoncanonicalInput::eDelete;
break;
case VK_ESCAPE:
canInput.type = ENoncanonicalInput::eEscape;
break;
case VK_END:
canInput.type = ENoncanonicalInput::eEnd;
break;
case VK_F1:
canInput.type = ENoncanonicalInput::eFunction1;
break;
case VK_F2:
canInput.type = ENoncanonicalInput::eFunction2;
break;
case VK_F3:
canInput.type = ENoncanonicalInput::eFunction3;
break;
case VK_F4:
canInput.type = ENoncanonicalInput::eFunction4;
break;
case VK_F5:
canInput.type = ENoncanonicalInput::eFunction5;
break;
case VK_F6:
canInput.type = ENoncanonicalInput::eFunction6;
break;
case VK_F7:
canInput.type = ENoncanonicalInput::eFunction7;
break;
case VK_F8:
canInput.type = ENoncanonicalInput::eFunction8;
break;
case VK_F9:
canInput.type = ENoncanonicalInput::eFunction9;
break;
case VK_F10:
canInput.type = ENoncanonicalInput::eFunction10;
break;
case VK_F11:
canInput.type = ENoncanonicalInput::eFunction11;
break;
case VK_F12:
canInput.type = ENoncanonicalInput::eFunction12;
break;
case VK_NUMPAD0:
case VK_NUMPAD1:
case VK_NUMPAD2:
case VK_NUMPAD3:
case VK_NUMPAD4:
case VK_NUMPAD5:
case VK_NUMPAD6:
case VK_NUMPAD7:
case VK_NUMPAD8:
case VK_NUMPAD9:
{
canInput.type = ENoncanonicalInput::eInput;
canInput.string = '0' + (record.Event.KeyEvent.wVirtualKeyCode - VK_NUMPAD0);
break;
}
case VK_MULTIPLY:
{
canInput.type = ENoncanonicalInput::eInput;
canInput.string = '*';
break;
}
case VK_ADD:
{
canInput.type = ENoncanonicalInput::eInput;
canInput.string = '+';
break;
}
case VK_SUBTRACT:
{
canInput.type = ENoncanonicalInput::eInput;
canInput.string = '-';
break;
}
case VK_DIVIDE:
{
canInput.type = ENoncanonicalInput::eInput;
canInput.string = '/';
break;
}
case VK_DECIMAL:
{
canInput.type = ENoncanonicalInput::eInput;
canInput.string = '.';
break;
}
default:
dBreak = true;
}
canInput.bIsControlSequence = record.Event.KeyEvent.dwControlKeyState & (RIGHT_CTRL_PRESSED | LEFT_CTRL_PRESSED);
canInput.bIsAltSequence = record.Event.KeyEvent.dwControlKeyState & (RIGHT_ALT_PRESSED | LEFT_ALT_PRESSED);
canInput.bIsShiftSequence = record.Event.KeyEvent.dwControlKeyState & (SHIFT_PRESSED);
// TODO: numpad
if (!dBreak || ((record.Event.KeyEvent.wVirtualKeyCode <= VK_HELP &&
record.Event.KeyEvent.wVirtualKeyCode != VK_SPACE &&
(record.Event.KeyEvent.wVirtualKeyCode < VK_NUMPAD0 || record.Event.KeyEvent.wVirtualKeyCode > VK_NUMPAD9))))
{
}
else
{
canInput.type = ENoncanonicalInput::eInput;
key = AuLocale::ConvertFromWChar(&record.Event.KeyEvent.uChar.UnicodeChar, 1);
if (key.empty())
{
return;
}
for (z = 0; z < record.Event.KeyEvent.wRepeatCount; z++)
{
key.resize(::strnlen(key.data(), key.size()));
canInput.string += key;
2022-08-13 23:00:08 +00:00
if (key.size())
{
dBreak = false;
}
}
}
break;
}
case MOUSE_EVENT:
{
dBreak = true;
if (record.Event.MouseEvent.dwEventFlags == MOUSE_WHEELED)
{
canInput.type = ENoncanonicalInput::eScroll;
canInput.iScrollDeltaY = AuStaticCast<AuInt16>(AuBitsToHigher((AuUInt32)record.Event.MouseEvent.dwButtonState));
if (canInput.iScrollDeltaY > 1)
canInput.iScrollDeltaY = 1;
else if (canInput.iScrollDeltaY < -1)
canInput.iScrollDeltaY = -1;
dBreak = false;
}
else if ((record.Event.MouseEvent.dwButtonState & FROM_LEFT_1ST_BUTTON_PRESSED) != 0)
{
ConsoleTTY::LeaveScrollMode();
}
else if ((record.Event.MouseEvent.dwButtonState & RIGHTMOST_BUTTON_PRESSED) != 0 && record.Event.MouseEvent.dwEventFlags != 1)
{
TryPasteFromClipboard();
}
else
{
// NOOP
}
break;
}
default:
{
dBreak = true;
break;
}
};
if (dBreak)
{
continue;
}
if (!AuTryInsert(gCanonicalBuffer, canInput))
{
return;
}
}
#else
//
#endif
}
void NoncanonicalTick()
{
ProcessCanonical(INVALID_HANDLE_VALUE);
}
void SignalKillNT()
{
if (IS_STREAM_HANDLE_VALID(gWin32Thread) &&
pCancelSynchronousIo)
{
pCancelSynchronousIo(gWin32Thread);
}
if (IS_STREAM_HANDLE_VALID(gTerminateConsole))
{
SetEvent(gTerminateConsole);
}
if (WaitForSingleObject(gWin32Thread, 200) != WAIT_OBJECT_0)
{
TerminateThread(gWin32Thread, 0);
}
AuWin32CloseHandle(gWin32Thread);
}
bool EnterNoncanonicalMode()
{
DWORD mode;
#if defined(AURORA_PLATFORM_WIN32)
if (!GetConsoleMode(gInputStream, &mode))
{
return false;
}
gCanonicalBackup = mode;
mode &= ~(ENABLE_LINE_INPUT | ENABLE_ECHO_INPUT | ENABLE_QUICK_EDIT_MODE);
mode |= ENABLE_PROCESSED_INPUT | ENABLE_MOUSE_INPUT | ENABLE_WINDOW_INPUT | ENABLE_EXTENDED_FLAGS;
SetConsoleMode(gInputStream, mode);
gCanonicalEnabled = true;
SignalKillNT();
INPUT_RECORD niceWorkMicrosoft[2] {};
niceWorkMicrosoft[0].EventType = KEY_EVENT;
niceWorkMicrosoft[0].Event.KeyEvent.bKeyDown = TRUE;
niceWorkMicrosoft[0].Event.KeyEvent.dwControlKeyState = 0;
niceWorkMicrosoft[0].Event.KeyEvent.wRepeatCount = 1;
niceWorkMicrosoft[0].Event.KeyEvent.wVirtualKeyCode = VK_RETURN;
niceWorkMicrosoft[0].Event.KeyEvent.uChar.UnicodeChar = '\r';
niceWorkMicrosoft[0].Event.KeyEvent.wVirtualScanCode = pMapVirtualKeyA(VK_RETURN, MAPVK_VK_TO_VSC);
niceWorkMicrosoft[1].EventType = KEY_EVENT;
niceWorkMicrosoft[1].Event.KeyEvent.bKeyDown = FALSE;
niceWorkMicrosoft[1].Event.KeyEvent.dwControlKeyState = 0;
niceWorkMicrosoft[1].Event.KeyEvent.wRepeatCount = 1;
niceWorkMicrosoft[1].Event.KeyEvent.wVirtualKeyCode = VK_RETURN;
niceWorkMicrosoft[1].Event.KeyEvent.uChar.UnicodeChar = '\r';
niceWorkMicrosoft[1].Event.KeyEvent.wVirtualScanCode = pMapVirtualKeyA(VK_RETURN, MAPVK_VK_TO_VSC);
DWORD idc;
pWriteConsoleInputW(gInputStream, niceWorkMicrosoft, 2, &idc);
pWriteConsoleInputW(gInputStream, niceWorkMicrosoft, 2, &idc);
return true;
#endif
}
void LeaveNoncanonicalMode()
{
#if defined(AURORA_PLATFORM_WIN32)
if (!AuExchange(gCanonicalEnabled, false))
{
return;
}
SetConsoleMode(gInputStream, gCanonicalBackup);
gWin32Thread = CreateThread(nullptr, 0, StdInWin32Thread, nullptr, 0, nullptr);
#endif
}
#else
struct termios gTermSavedAttributes;
bool EnterNoncanonicalMode()
{
struct termios tattr;
::tcgetattr(STDIN_FILENO, &gTermSavedAttributes);
::tcgetattr(STDIN_FILENO, &tattr);
::cfmakeraw(&tattr);
tattr.c_iflag |= BRKINT | IGNBRK;//~(BRKINT | IGNBRK );//ISIG | BRKINT | VINTR;
tattr.c_lflag |= ISIG ;//~(ISIG | IEXTEN );//ISIG | BRKINT | VINTR;
tattr.c_cc[VMIN] = 1;
tattr.c_cc[VTIME] = 0;
if (::tcsetattr(STDIN_FILENO, TCSANOW, &tattr))
{
return false;
}
gCanonicalUnixOn = true;
return true;
}
void LeaveNoncanonicalMode()
{
::tcsetattr(STDIN_FILENO, TCSANOW, &gTermSavedAttributes);
}
static void ProcessLinesSysCP();
static void FlushStdOut();
static bool AsyncReadAnyOrReadStreamBlock();
static bool InputStreamAvailable();
static AuUInt32 SyncReadConsole();
void NoncanonicalTick()
{
FlushStdOut();
if (!InputStreamAvailable())
{
return;
}
SyncReadConsole();
ProcessLinesSysCP();
}
#endif
bool IsStdOutTTY()
{
#if defined(AURORA_PLATFORM_WIN32)
//#if defined(AURORA_IS_MODERNNT_DERIVED)
HANDLE hConsole;
DWORD dwType;
hConsole = GetStdHandle(STD_OUTPUT_HANDLE);
if (!(dwType = GetFileType(hConsole)))
{
return false;
}
return dwType == FILE_TYPE_CHAR;
#else
return false;
//return IsStdOutTTY(STDOUT_FILENO);
#endif
}
bool IsStdOutTTY(AuUInt handle)
{
#if defined(AURORA_IS_MODERNNT_DERIVED)
HANDLE hConsole {(HANDLE)handle};
DWORD dwType;
if (handle == 0 || (AuUInt)INVALID_HANDLE_VALUE == handle)
{
return false;
}
if (!(dwType = GetFileType(hConsole)))
{
return false;
}
return dwType == FILE_TYPE_CHAR;
#else
return ::isatty((int)handle);
#endif
}
AuList<NoncanonicalInput> DequeueNoncanonicalInput()
{
AU_LOCK_GUARD(gRingLock);
if (gLoopSource)
{
gLoopSource->Reset();
}
return AuExchange(gCanonicalBuffer, {});
}
bool ConsoleHasDataLoopSource::IsSignaled()
{
#if defined(AURORA_IS_MODERNNT_DERIVED)
return gLineIndex || (gRuntimeConfig.console.enableStdPassthrough && gRuntimeConfig.console.enableStdIn && gEncodedIndex);
#else
return LSHandle::IsSignaled();
#endif
}
AuLoop::ELoopSource ConsoleHasDataLoopSource::GetType()
{
return AuLoop::ELoopSource::eProcessStdIn;
}
bool ConsoleHasDataLoopSource::OnTrigger(AuUInt handle)
{
#if defined(AURORA_IS_MODERNNT_DERIVED)
return IsSignaled();
#else
return true;
#endif
}
bool ConsoleHasDataLoopSource::WaitOn(AuUInt32 timeout)
{
return LSHandle::WaitOn(timeout);
}
#if !defined(AURORA_IS_MODERNNT_DERIVED)
const AuList<AuUInt> &ConsoleHasDataLoopSource::GetHandles()
{
return this->handles;
}
bool ConsoleHasDataLoopSource::Singular()
{
return false;
}
#endif
2023-12-18 01:42:50 +00:00
bool ConsoleSecondaryDataLoopSource::IsSignaled()
{
return LSHandle::IsSignaled() ||
gLineIndex;
}
AuLoop::ELoopSource ConsoleSecondaryDataLoopSource::GetType()
{
return AuLoop::ELoopSource::eProcessStdIn;
}
bool ConsoleSecondaryDataLoopSource::OnTrigger(AuUInt handle)
{
return true;
}
bool ConsoleSecondaryDataLoopSource::WaitOn(AuUInt32 timeout)
{
return LSHandle::WaitOn(timeout);
}
const AuList<AuUInt>& ConsoleSecondaryDataLoopSource::GetHandles()
{
return this->handles;
}
bool ConsoleSecondaryDataLoopSource::Singular()
{
return false;
}
static void StartLogger()
{
if (gRuntimeConfig.console.enableStdPassthrough == gRuntimeConfig.console.enableStdOut)
{
return;
}
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
Console::AddDefaultLogger(AuUnsafeRaiiToShared(&Logging::Sinks::gStdConsoleSink));
}
static bool gConsoleStarted = false;
2021-09-06 10:58:08 +00:00
void Start()
{
2022-01-19 17:08:13 +00:00
if (AuExchange(gConsoleStarted, true)) return;
2021-09-06 10:58:08 +00:00
#if defined(AURORA_IS_MODERNNT_DERIVED)
DWORD dwMode;
bool ok;
// Obtain a win32 file HANDLE of STDIN
auto fileHandle =
#if defined(AURORA_PLATFORM_WIN32)
GetStdHandle(STD_INPUT_HANDLE);
#else
Win32Open(L"CONIN$",
GENERIC_READ,
FILE_SHARE_READ,
false,
OPEN_EXISTING,
0,
0);
#endif
2021-09-06 10:58:08 +00:00
SysAssert(fileHandle != INVALID_HANDLE_VALUE, "Couldn't open CONIN");
gInputStream = fileHandle;
// Obtain a win32 file HANDLE of STDOUT
fileHandle =
#if defined(AURORA_PLATFORM_WIN32)
GetStdHandle(STD_OUTPUT_HANDLE);
#else
Win32Open(L"CONOUT$",
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE,
false,
OPEN_EXISTING,
0,
0);
#endif
2021-09-06 10:58:08 +00:00
SysAssert(fileHandle != INVALID_HANDLE_VALUE, "Couldn't open CONOUT");
gOutputStream = fileHandle;
bool isNotPipe = GetFileType(fileHandle) != FILE_TYPE_PIPE;
if (isNotPipe)
2021-09-06 10:58:08 +00:00
{
#if defined(AURORA_PLATFORM_WIN32)
// Get current console flags
if (GetConsoleMode(gOutputStream, &dwMode))
2021-09-06 10:58:08 +00:00
{
if (gRuntimeConfig.console.enableStdPassthrough ^ gRuntimeConfig.console.enableStdOut)
{
2022-12-14 01:35:18 +00:00
if (AuSwInfo::IsWindows10OrGreater() && AuSwInfo::GetPlatformInfo().uKernelPatch >= 10586)
{
dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
}
else
{
gSupportsColorOutput = false;
}
// Enable escape processing; enable colored output
2022-12-14 01:35:18 +00:00
dwMode |= ENABLE_PROCESSED_OUTPUT;
ok = SetConsoleMode(gOutputStream, dwMode);
SysAssert(ok, "Couldn't set console mode");
// Set the output stream to use UTF-8
ok = SetConsoleOutputCP(CP_UTF8);
SysAssert(ok, "Couldn't maintain UTF-8 stdout stream");
}
2021-09-06 10:58:08 +00:00
}
// Set binary mode if redirecting stdin to user
if (gRuntimeConfig.console.enableStdPassthrough && gRuntimeConfig.console.enableStdOut)
{
SetConsoleMode(gOutputStream, 0);
}
if (gRuntimeConfig.console.enableStdPassthrough ^ gRuntimeConfig.console.enableStdIn)
{
ok = SetConsoleCP(CP_UTF8);
#if 0
SysAssert(ok, "Couldn't maintain UTF-8 stdin stream");
#endif
}
#endif
2021-09-06 10:58:08 +00:00
}
if (gRuntimeConfig.console.enableStdPassthrough && gRuntimeConfig.console.enableStdIn && isNotPipe)
2021-09-06 10:58:08 +00:00
{
#if defined(AURORA_PLATFORM_WIN32)
2021-09-06 10:58:08 +00:00
if (!GetConsoleMode(gInputStream, &dwMode))
{
ok = SetConsoleMode(gInputStream, dwMode & ~(ENABLE_PROCESSED_INPUT | ENABLE_LINE_INPUT));
SysAssert(ok, "Couldn't maintain binary stdin stream");
}
#endif
2021-09-06 10:58:08 +00:00
}
//else
{
gTerminateConsole = CreateEvent(nullptr, true, false, nullptr);
gWin32Thread = CreateThread(nullptr, 0, StdInWin32Thread, nullptr, 0, nullptr);
}
2021-09-06 10:58:08 +00:00
#elif defined(AURORA_IS_POSIX_DERIVED)
gInputStream = STDIN_FILENO;
gOutputStream = STDOUT_FILENO;
#endif
2023-12-18 01:42:50 +00:00
gLoopSource = AuMakeSharedPanic<ConsoleHasDataLoopSource>(
#if !defined(AURORA_IS_MODERNNT_DERIVED)
gInputStream
#endif
);
2023-12-18 01:42:50 +00:00
gLoopSourceConTTY = AuMakeSharedPanic<ConsoleSecondaryDataLoopSource>(
#if defined(AURORA_IS_MODERNNT_DERIVED)
gLoopSource.get()
#else
gInputStream
#endif
);
2022-04-06 01:24:38 +00:00
SysAssert(gInputStream != DEFAULT_HANDLE_VAL, "Couldn't allocate input stream handler");
SysAssert(gOutputStream != DEFAULT_HANDLE_VAL, "Couldn't allocate output stream handler");
2021-09-06 10:58:08 +00:00
StartLogger();
2021-09-06 10:58:08 +00:00
}
void Init()
{
#if defined(AURORA_PLATFORM_WIN32)
if (GetConsoleWindow() == NULL)
{
if (!gRuntimeConfig.console.enableConsole)
2021-09-06 10:58:08 +00:00
{
return;
}
if (gRuntimeConfig.console.forceConsoleWindow)
2021-09-06 10:58:08 +00:00
{
auto ok = AllocConsole();
SysAssert(ok, "Request of a Win32 console yielded nada, forceConsole wasn't respected");
2021-09-06 10:58:08 +00:00
}
}
else
{
// Under windows, applications linked with the console subsystem flag should have output no matter what disableAll/forceConsoleWindow says
// The amount of effort it takes to link as console app for a conhost window far exceeds whatever 'forceConsoleWindow' is defined as
// One does not simply switch over to conapp for the fun of it. We can't or at least shouldn't destroy or hide conhost with hacks
// Assuming stdconsole is wanted - DO NOT RETURN - LINK AS A WINDOWED APP IF YOU DO NOT WANT A CONSOLE WINDOW
}
#elif defined(AURORA_IS_POSIX_DERIVED)
// no always means no under UNIX targets
// we *know* stdin/out will be available under these standards, dont fuck with what might be an IPC channel
if (!gRuntimeConfig.console.enableConsole)
2021-09-06 10:58:08 +00:00
{
return;
}
#endif
Start();
}
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
static AuUInt32 WriteStdOutBlocking(const void *data, AuUInt32 length)
2021-09-06 10:58:08 +00:00
{
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
if (gBufferMode)
{
return gStdoutBuffer.Write(data, length);
}
2021-09-06 10:58:08 +00:00
if (!IS_STREAM_HANDLE_VALID(gOutputStream))
{
return 0;
}
if (length == 0)
{
return {};
}
2021-09-06 10:58:08 +00:00
#if defined(IO_POSIX_STREAMS)
AuInt32 written = ::write(gOutputStream, data, length);
2021-09-06 10:58:08 +00:00
if (written < 0)
{
return 0;
}
#elif defined(AURORA_IS_MODERNNT_DERIVED)
DWORD written;
if (!::WriteFile(gOutputStream, data, length, &written, NULL))
2021-09-06 10:58:08 +00:00
{
return 0;
}
#endif
return written;
}
void FlushStdOutNb()
{
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
if (gBufferMode)
{
return;
}
AuUInt32 written {};
written = WriteStdOutBlocking(gStdoutBuffer.begin(), gStdoutBuffer.size());
gStdoutBuffer.clear();
#if defined(AURORA_IS_POSIX_DERIVED)
::fsync(gOutputStream);
#endif
}
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
AuUInt32 WriteStdOut(const void *data, AuUInt32 length)
{
AU_LOCK_GUARD(gRingLock);
return gStdoutBuffer.Write(data, length);
}
2022-05-03 04:31:49 +00:00
AuUInt32 WriteStdOutBlocking2(const void *data, AuUInt32 length)
{
AU_LOCK_GUARD(gRingLock);
2022-05-03 04:31:49 +00:00
return WriteStdOutBlocking(data, length);
}
void Lock()
{
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
AU_LOCK_GUARD(gRingLock);
gBufferMode = true;
}
void Unlock()
{
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
AU_LOCK_GUARD(gRingLock);
gBufferMode = false;
FlushStdOutNb();
}
AuUInt32 WriteStdOutNonBlocking(const void *data, AuUInt32 length)
{
return gStdoutBuffer.Write(data, length);
}
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
static void FlushStdOut()
{
AU_LOCK_GUARD(gRingLock);
FlushStdOutNb();
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
}
void WriteStdOut(AuUInt8 level, const ConsoleMessage &msg, const AuString &formatted)
{
2022-12-14 01:35:18 +00:00
bool bUseAlt {};
auto writeLine = formatted;
2022-12-14 01:35:18 +00:00
if (!gSupportsColorOutput)
{
if (IsStdOutTTY())
{
bUseAlt = true;
}
}
else
{
const char *pAnsiCode =
2024-03-10 09:45:03 +00:00
static_cast<EAnsiColor>(msg.color) < EAnsiColor::eEnumCount ?
2022-12-14 01:35:18 +00:00
kAnsiColorForegroundToVirtualEscape[static_cast<AuUInt>(msg.color)] :
"";
writeLine = pAnsiCode + writeLine + kAnsiColorForegroundToVirtualEscape[static_cast<AuUInt>(EAnsiColor::eReset)];
}
#if defined(AURORA_IS_MODERNNT_DERIVED)
writeLine += '\r';
#endif
writeLine += '\n';
2022-12-14 01:35:18 +00:00
if (bUseAlt)
{
ConsoleTTY::TTYWrite(writeLine.c_str(), msg.color);
return;
}
#if defined(IO_POSIX_STREAMS)
if (Locale::GetInternalCodePage() == Locale::ECodePage::eUTF8)
{
WriteStdOut(writeLine.data(), writeLine.size());
}
else
{
AuString slow;
slow.resize(writeLine.size() * 4);
auto len = Locale::Encoding::EncodeUTF8(writeLine, Aurora::Memory::MemoryViewWrite {slow.data(), slow.size()}, Locale::ECodePage::eSysUnk);
if (len.first != 0)
{
WriteStdOut(slow.data(), len.second);
}
else
{
// better write this than nothing
WriteStdOut(writeLine.data(), writeLine.size());
}
}
#elif defined(AURORA_IS_MODERNNT_DERIVED)
WriteStdOut(writeLine.data(), AuUInt32(writeLine.size()));
#endif
}
2021-09-06 10:58:08 +00:00
static bool InputStreamAvailable()
{
#if defined(IO_POSIX_STREAMS)
timeval tv {};
fd_set fds {};
FD_ZERO(&fds);
FD_SET(gInputStream, &fds);
select(gInputStream + 1, &fds, NULL, NULL, &tv);
return (FD_ISSET(0, &fds));
#elif defined(AURORA_IS_MODERNNT_DERIVED)
// Inline non-blocking is not legal on windows.
// One can force it with the overlapped flag, seemingly valid since win8, but defined as illegal since the 90s
2021-09-06 10:58:08 +00:00
return false;
#else
return false;
#endif
}
AuUInt32 ReadStdIn(void *data, AuUInt32 length)
{
Pump();
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
{
AU_LOCK_GUARD(gRingLock);
2021-09-06 10:58:08 +00:00
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
auto readable = AuMin(AuUInt32(length), AuUInt32(gEncodedIndex));
2021-09-06 10:58:08 +00:00
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
AuMemcpy(data, gLineEncodedBuffer, readable);
2021-09-06 10:58:08 +00:00
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
const auto remainingBytes = gEncodedIndex - readable;
if (remainingBytes)
{
AuMemmove(gLineEncodedBuffer, &gLineEncodedBuffer[readable], remainingBytes);
}
gEncodedIndex = remainingBytes;
if (gLoopSource && !gEncodedIndex)
{
gLoopSource->Reset();
}
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
return readable;
}
2021-09-06 10:58:08 +00:00
}
static void ProcessLines(AuList<AuString> &lines)
2021-09-06 10:58:08 +00:00
{
AuMach index = 0, startIdx = 0;
if (gLineIndex == 0)
{
return;
}
auto end = gLineBuffer.data() + gLineIndex;
while (true)
{
auto a = std::find(gLineBuffer.data() + startIdx, end, '\n');
if (a == end) break;
index = a - gLineBuffer.data();
auto line = gLineBuffer.substr(startIdx, index - startIdx);
startIdx = index + 1;
if (line[line.size() - 1] == '\r')
{
line.pop_back();
}
if (line.size())
{
lines.push_back(line);
2021-09-06 10:58:08 +00:00
}
}
if (index != 0)
{
const auto remainingBytes = gLineIndex - startIdx;
if (remainingBytes)
{
2022-01-19 17:08:13 +00:00
AuMemmove(gLineBuffer.data(), &gLineBuffer.data()[startIdx], remainingBytes);
2021-09-06 10:58:08 +00:00
gLineIndex -= startIdx;
}
else
{
gLineIndex = 0;
}
2021-09-06 10:58:08 +00:00
}
}
static void CanFlushLines()
2021-09-06 10:58:08 +00:00
{
AuList<AuString> binaryLines;
2021-09-06 10:58:08 +00:00
gLineBuffer[gLineIndex++] = '\n';
ProcessLines(binaryLines);
for (const auto &str : binaryLines)
{
NoncanonicalInput canInput {};
canInput.type = ENoncanonicalInput::eInput;
canInput.string += str;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
}
// TODO (Reece): This is disgusting.
// I need to rewrite this frame
static void ConsumeCanInputUnixEsc()
{
static Locale::Encoding::TextStreamEncoder stream(Locale::GetInternalCodePage());
2021-09-06 10:58:08 +00:00
{
AU_LOCK_GUARD(gRingLock);
int i {};
for (i = 0;
i < gEncodedIndex;
)
{
auto c = gLineEncodedBuffer[i];
NoncanonicalInput canInput {};
canInput.iScrollDeltaY = 0;
canInput.bIsControlSequence = canInput.bIsControlSequence = canInput.bIsControlSequence = false;
if (c == 127)
{
CanFlushLines();
canInput.type = ENoncanonicalInput::eBackspace;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (c == '\n' || c == 13)
{
CanFlushLines();
canInput.type = ENoncanonicalInput::eEnter;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if ((c == 27 && gEncodedIndex == 1))
{
CanFlushLines();
canInput.type = ENoncanonicalInput::eEscape;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (c == '\t')
{
CanFlushLines();
canInput.type = ENoncanonicalInput::eTab;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (c == '\r')
{
CanFlushLines();
}
else if (c < 27)
{
CanFlushLines();
canInput.type = ENoncanonicalInput::eInput;
canInput.string = 'a' + c - 1;
canInput.bIsControlSequence = true;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (c == 27)
{
CanFlushLines();
i++;
c = gLineEncodedBuffer[i];
if (c == '[')
{
i++;
auto esc = gLineEncodedBuffer[i];
if (esc == 'A')
{
canInput.type = ENoncanonicalInput::eArrowUp;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (esc == 'B')
{
canInput.type = ENoncanonicalInput::eArrowDown;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (esc == 'D')
{
canInput.type = ENoncanonicalInput::eArrowLeft;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (esc == 'C')
{
canInput.type = ENoncanonicalInput::eArrowRight;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
#define HANDLE_ALTS_OF_NEXT_OP(nestedChar, op) \
if (next == nestedChar) \
{ \
canInput.type = ENoncanonicalInput::op; \
SysAssert(AuTryInsert(gCanonicalBuffer, canInput)); \
} \
else if (next == ';') \
{ \
i++; \
auto esc = gLineEncodedBuffer[i]; \
if (esc == '5') \
{ \
i++; \
auto next = gLineEncodedBuffer[i]; \
\
if (next == nestedChar) \
{ \
canInput.type = ENoncanonicalInput::op; \
canInput.bIsControlSequence = true; \
SysAssert(AuTryInsert(gCanonicalBuffer, canInput)); \
} \
} \
else if (esc == '6') \
{ \
i++; \
auto next = gLineEncodedBuffer[i]; \
\
if (next == nestedChar) \
{ \
canInput.type = ENoncanonicalInput::op; \
canInput.bIsShiftSequence = true; \
canInput.bIsControlSequence = true; \
SysAssert(AuTryInsert(gCanonicalBuffer, canInput)); \
} \
} \
else if (esc == '2') \
{ \
i++; \
auto next = gLineEncodedBuffer[i]; \
\
if (next == nestedChar) \
{ \
canInput.type = ENoncanonicalInput::op; \
canInput.bIsShiftSequence = true; \
SysAssert(AuTryInsert(gCanonicalBuffer, canInput)); \
} \
} \
else if (esc == '8') \
{ \
i++; \
auto next = gLineEncodedBuffer[i]; \
\
if (next == nestedChar) \
{ \
canInput.type = ENoncanonicalInput::op; \
canInput.bIsShiftSequence = true; \
canInput.bIsControlSequence = true; \
canInput.bIsAltSequence = true; \
SysAssert(AuTryInsert(gCanonicalBuffer, canInput)); \
} \
} \
}
#define HANDLE_PAGE_OPS(no, op) \
else if (esc == no) \
{ \
i++; \
auto next = gLineEncodedBuffer[i]; \
HANDLE_ALTS_OF_NEXT_OP('~', op) \
}
HANDLE_PAGE_OPS('6', ePageDown)
HANDLE_PAGE_OPS('5', ePageUp)
HANDLE_PAGE_OPS('2', eInsert)
HANDLE_PAGE_OPS('3', eDelete)
else if (esc == '1')
{
i++;
auto next = gLineEncodedBuffer[i];
if (next == ';')
{
i++;
auto esc = gLineEncodedBuffer[i];
if (esc == '5')
{
i++;
auto esc = gLineEncodedBuffer[i];
if (esc == 'A')
{
canInput.type = ENoncanonicalInput::eScroll;
canInput.iScrollDeltaY = 4;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
else if (esc == 'B')
{
canInput.type = ENoncanonicalInput::eScroll;
canInput.iScrollDeltaY = -4;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
}
}
}
}
else if (c == '?')
{
i++;
auto esc = gLineEncodedBuffer[i];
}
#if 0
else if (c == ';')
{
i++;
auto next = gLineEncodedBuffer[i];
if (next == '~')
{
canInput.type = ENoncanonicalInput::ePageUp;
SysAssert(AuTryInsert(gCanonicalBuffer, canInput));
}
}
#endif
}
else
{
gLineBuffer[gLineIndex++] = c;
}
i++;
}
CanFlushLines();
2021-09-06 10:58:08 +00:00
{
const auto remainingBytes = gEncodedIndex - i;
if (remainingBytes)
{
AuMemmove(gLineEncodedBuffer, &gLineEncodedBuffer[i], remainingBytes);
}
gEncodedIndex = remainingBytes;
2021-09-06 10:58:08 +00:00
}
}
}
2021-09-06 10:58:08 +00:00
static void ProcessLinesSysCP()
{
#if defined(AURORA_IS_MODERNNT_DERIVED)
static Locale::Encoding::TextStreamEncoder stream(Locale::ECodePage::eUTF8);
#else
static Locale::Encoding::TextStreamEncoder stream(Locale::GetInternalCodePage());
#endif
2021-09-06 10:58:08 +00:00
AuList<AuString> lines;
{
if (gCanonicalUnixOn)
{
if (gLoopSource)
{
gLoopSource->Set();
}
ConsumeCanInputUnixEsc();
}
else
{
auto ret = stream.DecodeUTF8(gLineEncodedBuffer, gEncodedIndex, gLineBuffer.data() + gLineIndex, AuUInt32(gLineBuffer.size() - gLineIndex));
// increment backline buffer
{
const auto remainingBytes = gEncodedIndex - ret.first;
if (remainingBytes)
{
AuMemmove(gLineEncodedBuffer, &gLineEncodedBuffer[ret.first], remainingBytes);
}
gEncodedIndex = remainingBytes;
}
// increment frontline buffer
{
gLineIndex += ret.second;
}
if (gLoopSource)
{
gLoopSource->Set();
}
ProcessLines(lines);
}
2021-09-06 10:58:08 +00:00
}
if (lines.size())
{
for (const auto &line : lines)
{
Console::DispatchRawLine(line);
}
ConsoleTTY::OnEnter();
}
2021-09-06 10:58:08 +00:00
}
static AuUInt32 SyncReadConsole()
{
void *data = &gLineEncodedBuffer[gEncodedIndex];
auto length = kLineBufferMax - gEncodedIndex;
if (!IS_STREAM_HANDLE_VALID(gInputStream))
{
return 0;
}
if (length == 0)
{
return 0;
}
#if defined(AURORA_IS_MODERNNT_DERIVED)
DWORD read = length;
if (!ReadFile(gInputStream, data, read, &read, NULL))
{
return 0;
}
gRingLock->Lock();
2021-09-06 10:58:08 +00:00
gEncodedIndex += read;
gRingLock->Unlock();
2021-09-06 10:58:08 +00:00
return read;
#elif defined(IO_POSIX_STREAMS)
auto bread = ::read(gInputStream, data, length);
if (bread < 0)
{
return 0;
}
gRingLock->Lock();
2021-09-06 10:58:08 +00:00
gEncodedIndex += bread;
gRingLock->Unlock();
2021-09-06 10:58:08 +00:00
return bread;
#else
return 0;
#endif
}
static bool AsyncReadAnyOrReadStreamBlock()
{
if (!SyncReadConsole())
{
return false;
}
if (gRuntimeConfig.console.enableStdPassthrough && gRuntimeConfig.console.enableStdIn)
2021-09-06 10:58:08 +00:00
{
if (gLoopSource)
{
gLoopSource->Set();
}
2021-09-06 10:58:08 +00:00
return true;
}
ProcessLinesSysCP();
return true;
2021-09-06 10:58:08 +00:00
}
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
void Flush()
{
FlushStdOut();
}
2021-09-06 10:58:08 +00:00
void Pump()
{
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
FlushStdOut();
if (!gRuntimeConfig.console.enableStdPassthrough && !gRuntimeConfig.console.enableStdIn)
2021-09-06 10:58:08 +00:00
{
return;
}
if (!IS_STREAM_HANDLE_VALID(gInputStream))
{
return;
}
if (!InputStreamAvailable())
{
return;
}
AsyncReadAnyOrReadStreamBlock();
}
void Exit()
{
if (IS_STREAM_HANDLE_VALID(gTerminateConsole))
{
#if defined(AURORA_IS_MODERNNT_DERIVED)
SetEvent(gTerminateConsole);
#endif
}
#if defined(AURORA_IS_MODERNNT_DERIVED)
gCanonicalEnabled = false;
Pump();
::FlushFileBuffers(gOutputStream);
SignalKillNT();
2021-09-06 10:58:08 +00:00
2021-09-06 10:58:08 +00:00
AuWin32CloseHandle(gTerminateConsole);
// Note: CloseHandle in the middle of a ReadFile blocks
#if !defined(AURORA_PLATFORM_WIN32)
2021-09-06 10:58:08 +00:00
AuWin32CloseHandle(gInputStream);
AuWin32CloseHandle(gOutputStream);
#endif
gConsoleStarted = false;
2021-09-06 10:58:08 +00:00
#endif
2023-12-18 01:42:50 +00:00
AuResetMember(gLoopSource);
AuResetMember(gLoopSourceConTTY);
}
2023-12-18 01:42:50 +00:00
AuLoop::ILSEvent* GetConTTYEvent()
{
return gLoopSourceConTTY.get();
}
AuSPtr<AuLoop::ILoopSource> GetLoopSource()
{
2023-12-18 01:42:50 +00:00
#if defined(AURORA_IS_MODERNNT_DERIVED)
if (gCanonicalEnabled)
{
return gLoopSourceConTTY;
}
#endif
return gLoopSource;
}
2023-12-18 01:42:50 +00:00
AuSPtr<AuLoop::ILoopSource> GetLoopSourceConTTY()
{
return gLoopSourceConTTY;
}
2021-09-06 10:58:08 +00:00
#else
void Pump()
{
}
void Init()
{
}
void Exit()
{
}
void Start()
{
}
[*/+/-] MEGA COMMIT. ~2 weeks compressed. The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api. [+] AuMakeSharedArray [+] Technet ArgvQuote [+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.) [+] auEndianness -> Endian swap utils [+] AuGet<N>(...) [*] AUE_DEFINE conversion for ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump [+] ConsoleMessage ByteBuffer serialization [+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks [*] Split logger from console subsystem [+] StartupParameters -> A part of a clean up effort under Process [*] Refactor SysErrors header + get caller hack [+] Atomic APIs [+] popcnt [+] Ring Buffer sink [+] Added more standard errors Catch, Submission, LockError, NoAccess, ResourceMissing, ResourceLocked, MalformedData, InSandboxContext, ParseError [+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace [+] IExitSubscriber, ETriggerLevel [*] Write bias the high performance RWLockImpl read-lock operation operation [+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem) [*] Updated API style Digests [+] CpuId::CpuBitCount [+] GetUserProgramsFolder [+] GetPackagePath [*] Split IStreamReader with an inl file [*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer [+] ICharacterProvider [+] ICharacterProviderEx [+] IBufferedCharacterConsumer [+] ProviderFromSharedString [+] ProviderFromString [+] BufferConsumerFromProvider [*] Parse Subsystem uses character io bufferer [*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore] [+] ByteBuffer::ResetReadPointer [*] Bug fix bytebuffer base not reset on free and some scaling issues [+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section [*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing [+] Added 64 *byte* fast RNG seeds [+] File Advisorys/File Lock Awareness [+] Added extended IAuroraThread from OS identifier caches for debug purposes [*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions. [*] Broke AuroraUtils/Typedefs out into a separate library [*] Update build script [+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media [*] Improved public API documentation [*] Bug fix `SetConsoleCtrlHandler` [+] Locale TimeDateToFileNameISO8601 [+] Console config stdOutShortTime [*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl) [*] Bug fixes in decoders [*] Major bug fix, AuMax [+] RateLimiter [+] Binary file sink [+] Log directory sink [*] Data header usability (more operators) [+] AuRemoveRange [+] AuRemove [+] AuTryRemove [+] AuTryRemoveRange [+] auCastUtils [+] Finish NewLSWin32Source [+] AuTryFindByTupleN, AuTryRemoveByTupleN [+] Separated AuRead/Write types, now in auTypeUtils [+] Added GetPosition/SetPosition to FileWriter [*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp [*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position) [*] Hack back in the sched deinit [+] File AIO loop source interop [+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw [+] Stub code for networking [+] Compression BaseStream/IngestableStreamBase [*] Major: read/write locks now support write-entrant read routines. [*] Compression subsystem now uses the MemoryView concept [*] Rewrite the base stream compressions, made them less broken [*] Update hashing api [*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing [+] Added new AuByteBuffer apis Trim, Pad, WriteFrom, WriteString, [TODO: ReadString] [+] Added ByteBufferPushReadState [+] Added ByteBufferPushWriteState [*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16. [*] ELogLevel is now an Aurora enum [+] Raised arbitrary limit in header to 255, the max filter buffer [+] Explicit GZip support [+] Explicit Zip support [+] Added [some] compressors et al
2022-02-17 00:11:40 +00:00
void Flush()
{
}
AuSPtr<AuLoop::ILoopSource> GetLoopSource()
{
return {};
}
2023-12-18 01:42:50 +00:00
AuSPtr<AuLoop::ILoopSource> GetLoopSourceConTTY()
{
return {};
}
AuLoop::ILSEvent* GetConTTYEvent()
{
return {};
}
2021-09-06 10:58:08 +00:00
#endif
}