AuroraRuntime/Include/Aurora/Loop/ILoopQueue.hpp
Reece cf70f0d45c [*/+/-] MEGA COMMIT. ~2 weeks compressed.
The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api.

[+] AuMakeSharedArray
[+] Technet ArgvQuote
[+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.)
[+] auEndianness -> Endian swap utils
[+] AuGet<N>(...)
[*] AUE_DEFINE conversion for
        ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump
[+] ConsoleMessage ByteBuffer serialization
[+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks
[*] Split logger from console subsystem
[+] StartupParameters -> A part of a clean up effort under Process
[*] Refactor SysErrors header + get caller hack
[+] Atomic APIs
[+] popcnt
[+] Ring Buffer sink
[+] Added more standard errors
        Catch,
        Submission,
        LockError,
        NoAccess,
        ResourceMissing,
        ResourceLocked,
        MalformedData,
        InSandboxContext,
        ParseError

[+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace
[+] IExitSubscriber, ETriggerLevel
[*] Write bias the high performance RWLockImpl read-lock operation operation
[+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem)
[*] Updated API style
        Digests
[+] CpuId::CpuBitCount
[+] GetUserProgramsFolder
[+] GetPackagePath
[*] Split IStreamReader with an inl file
[*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer
[+] ICharacterProvider
[+] ICharacterProviderEx
[+] IBufferedCharacterConsumer
[+] ProviderFromSharedString
[+] ProviderFromString
[+] BufferConsumerFromProvider
[*] Parse Subsystem uses character io bufferer
[*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore]
[+] ByteBuffer::ResetReadPointer
[*] Bug fix bytebuffer base not reset on free and some scaling issues
[+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section
[*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing
[+] Added 64 *byte* fast RNG seeds
[+] File Advisorys/File Lock Awareness
[+] Added extended IAuroraThread from OS identifier caches for debug purposes
[*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions.
[*] Broke AuroraUtils/Typedefs out into a separate library
[*] Update build script
[+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media
[*] Improved public API documentation
[*] Bug fix `SetConsoleCtrlHandler`
[+] Locale TimeDateToFileNameISO8601
[+] Console config stdOutShortTime
[*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl)
[*] Bug fixes in decoders
[*] Major bug fix, AuMax
[+] RateLimiter
[+] Binary file sink
[+] Log directory sink
[*] Data header usability (more operators)
[+] AuRemoveRange
[+] AuRemove
[+] AuTryRemove
[+] AuTryRemoveRange
[+] auCastUtils
[+] Finish NewLSWin32Source
[+] AuTryFindByTupleN, AuTryRemoveByTupleN
[+] Separated AuRead/Write types, now in auTypeUtils
[+] Added GetPosition/SetPosition to FileWriter
[*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp
[*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position)
[*] Hack back in the sched deinit
[+] File AIO loop source interop
[+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw
[+] Stub code for networking
[+] Compression BaseStream/IngestableStreamBase
[*] Major: read/write locks now support write-entrant read routines.
[*] Compression subsystem now uses the MemoryView concept
[*] Rewrite the base stream compressions, made them less broken
[*] Update hashing api
[*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing
[+] Added new AuByteBuffer apis
    Trim, Pad, WriteFrom, WriteString, [TODO: ReadString]
[+] Added ByteBufferPushReadState
[+] Added ByteBufferPushWriteState
[*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16.
[*] ELogLevel is now an Aurora enum
[+] Raised arbitrary limit in header to 255, the max filter buffer
[+] Explicit GZip support
[+] Explicit Zip support
[+] Added [some] compressors

et al
2022-02-17 00:11:40 +00:00

186 lines
8.6 KiB
C++

/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: ILoopQueue.hpp
Date: 2022-2-12
Author: Reece
***/
#pragma once
namespace Aurora::Loop
{
// This is like a CF RunLoop with the input parameters of NT's WaitMultipleObjects
// This shouldn't be too much heavier than CF's libevent/NT abstraction in the style of a kevent interface with objects
struct ILoopQueue
{
/**
* @brief
* [*] Source add/remove operations should be followed by a Commit() to commit the changes
*
* [*] Calling Source[Add/Remove] during a Wait[Any/All[Ex]] will cause the wait function to
* return false, allowing for the caller to rewait on the updated queue, or to allow
* the thread to account for the object-was-invalidated condition
*
* @param source
* @return
*/
virtual bool SourceAdd(const AuSPtr<ILoopSource> &source) = 0;
/**
* @brief Same behaviour as SourceAdd
* @param source
* @param timeoutMS
* @return
*/
virtual bool SourceAddWithTimeout(const AuSPtr<ILoopSource> &source, AuUInt32 timeoutMS) = 0;
/**
* @brief Remove operations include commit. You do not need to follow a remove with a commit.
* Sources are defacto poped unless subscriber returns false indicating repeated lock attempts are wanted.
* Should no subscriber be registered, the loop source will not be automatically removed
* @param source
*/
virtual bool SourceRemove(const AuSPtr<ILoopSource> &source) = 0;
/**
* @brief Updates the OS watchdog list cache concept after Source[Remove/Add[WithTimeout]]
* @return
*/
virtual bool Commit() = 0;
/**
* @brief
* @return the amount of loop sources added to the queue
*/
virtual AuUInt32 GetSourceCount() = 0;
/**
* @brief Appends a callback handler to be paired with a loop source
*
* Once a loop source become signaled, and a Wait[Any/All] function call is made, loop source subscribers are called
* to filter the final should-remove operation. The final popped objects are returned from Wait[Any[Ex]] and forgotten.
*
* If no ILoopSourceSubscribers are registered, the edge case behaviour is that loop sources will not be automatically
* dequeued. No user code explicitly told us to do anything other than wait, so the default behaviour of Wait[Any[Ex]]
* shall be to not evict loop sources, until at least one callback is registered, or until SourceRemove is called.
*
* Returning true from a ILoopSourceSubscriber will allow the ILoopQueue to evict the loop source
* Returning false from a ILoopSourceSubscriber will prevent eviction, resulting in continued ILoopSourceSubscriber callbacks to all subscribers
*
* @param source
* @param subscriber
* @return
*/
virtual bool AddCallback(const AuSPtr<ILoopSource> &source, const AuSPtr<ILoopSourceSubscriber> &subscriber) = 0;
/**
* @brief Identical behaviour to that of the basic method. In addition, ILoopSourceSubscriberEx::OnTimeout is called
* on violation of the timeout provided to SourceAddWithTimeout
* @param source
* @param subscriber
* @return
*/
virtual bool AddCallbackEx(const AuSPtr<ILoopSource> &source, const AuSPtr<ILoopSourceSubscriberEx> &subscriber) = 0;
/**
* @brief Near identical behaviour of the extended AddCallback method.
* Registers a callback to handle all loop source signaled events.
* @param subscriber
* @return
*/
virtual bool AddCallback(const AuSPtr<ILoopSourceSubscriber> &subscriber) = 0;
/**
* @brief Indicates whether or not all the loop sources are in a signaled state and all
* relevant interfaces, such as internal IPC handlers and ILoopSourceSubscribers,
* have received the update. Merely waiting for all loop sources to finish is not
* sufficient. One must process the signaled states by calling a Wait function
* when the HasAnyPending() condition is valid.
* @return
*/
virtual bool HasFinished() = 0;
/**
* @brief Nonblocking wait-any for all objects in the loop queue
* @return
*/
virtual bool IsSignaled() = 0;
/**
* @brief Waits on all the submitted loop sources until they are all complete or until the timeout has finished.
* Note: the completion of another Wait[All/Any[Ex]] call may result in a
* @param timeout
* @return
*/
virtual bool WaitAll (AuUInt32 timeout = 0) = 0;
/**
* @brief Waits on all the loop sources until at least one is signaled
* @param timeout
* @return
*/
virtual AuUInt32 WaitAny (AuUInt32 timeout = 0) = 0;
/**
* @brief
* @param timeout
* @return
*/
virtual AuList<AuSPtr<ILoopSource>> WaitAnyEx(AuUInt32 timeout = 0) = 0;
/**
* @brief
* Mostly relevant on Windows where the limit is 64 versus horrible BSD-y socket
* optimized interfaces that can handle thousands of fds
*
* In large loop queues, a maximum of
* `sectionTickTime * (sectionDequeCount + clamp(platformSanity, sectionDequeCount))`
* kernel objects can be checked within a given time frame.
*
* What this effectively does is iterate over a subdivided work queue, aligned
* to platformSanity, [i, i+platform sanity] objects can be safely waitany on.
*
* On timeout of sectionTickTime,
* [i, i+sectionDequeCount] of the tick could be checked, however, we know
* [i, i+platform sanity] timeouted so, in the event of a timeout, we can
* asynchronously check [i+platform sanity, ...] to be nicer faster. Since
* we've already spent the maximum allocated time to poll on a mere subset,
* we may as well desperately check sectionDequeCount worth of objects, and
* write-off the time spent blocking. That way, we get upto platform sanity
* of is signaled checks for free in the worst case scenario. The perspective
* that the overshoot is expensive doesn't make sense, when the is signaled
* check is basically free and the timeout on a subset of the req was the
* bottleneck.
*
* Otherwise on success, we know to check in the range of [i, platform sanity
* for at least one alerted object.
*
* Looking anywhere else given any help from a waitany interface would be stupid.
*
* Internal logic will dequeue platform coefficient aligned handles, and the timeout
* optimization shall check [i+platform sanity, i+platform sanity+sectionDequeCount].
* 1 handle =/= 1 loop source, tho usually 1 loop source ~= 1 handle
*
* tl'dr:
* [*] dont use if you cant stall for sectionTickTime if sectionDequeCount of your
* loop sources are unsignaled
* [*] chug paths are intended for observer threads and not for maintaining high
* performance. no sane traditional application would ever have 64+ signalable
* objects stuck in a loop. though if you do, and you dont configure anything,
* the worst case scenario is, WaitAny can take upto a millisecond to discover
* a signaled loop source.
*
* @param sectionTickTime
* @param sectionDequeCount
*/
virtual void ChugPathConfigure(AuUInt32 sectionTickTime, AuSInt sectionDequeCount) = 0;
/**
* @brief Hints that the calling program understands the kernel shouldnt schedule tne entire source list, and instead, we should long poll
*/
virtual void ChugHint(bool value) = 0;
};
AUKN_SYM AuSPtr<ILoopQueue> NewLoopQueue();
}