AuroraRuntime/Source/IO/Loop/LoopQueue.Linux.hpp

138 lines
4.4 KiB
C++
Raw Normal View History

2022-04-05 05:59:23 +00:00
/***
Copyright (C) 2022 J Reece Wilson (a/k/a "Reece"). All rights reserved.
File: LoopQueue.Linux.hpp
Date: 2022-4-4
Author: Reece
***/
#pragma once
#include "ILoopSourceEx.hpp"
#include "LSEvent.hpp"
2022-04-05 05:59:23 +00:00
namespace Aurora::IO::Loop
2022-04-05 05:59:23 +00:00
{
struct LoopQueue : ILoopQueue
{
LoopQueue();
~LoopQueue();
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
bool AddHook(const AuFunction<void()> &func);
bool Init();
void Deinit();
2022-04-05 05:59:23 +00:00
bool SourceAdd(const AuSPtr<ILoopSource> &source) override;
bool SourceAddWithTimeout(const AuSPtr<ILoopSource> &source, AuUInt32 ms) override;
[+] Network + Protocol + TLS - Initial Commit ============================================================================= Network ]==================================================================== ============================================================================= [+] Added (very) early Aurora::IO::Net implementation [+] AuNet::EHostnameType [+] AuNet::EIPProtocol [+] AuNet::ENetworkError [+] AuNet::ETransportProtocol [+] AuNet::INetInterface [+] AuNet::INetSrvDatagram [+] AuNet::INetSrvResolve [+] AuNet::INetSrvSockets [+] AuNet::INetSrvWorkers [+] AuNet::INetWorker [+] AuNet::IPAddress [+] AuNet::IResolver [+] AuNet::ISocket [+] AuNet::IResolver [+] AuNet::ISocketBase [+] AuNet::ISocketChannel [+] AuNet::ISocketDriver [+] AuNet::ISocketDriverFactory [+] AuNet::ISocketServer [+] AuNet::ISocketServerDriver [+] AuNet::NetEndpoint [+] AuNet::NetError [+] AuNet::NetHostname (+implementation) ============================================================================= Protocol ]=================================================================== ============================================================================= [+] IProtocolInterceptor [+] IProtocolInterceptorEx [+] IProtocolStack (+implementation) ============================================================================= TLS ]======================================================================== ============================================================================= [+] ITLSContext [+] TLSProtocolRecv [+] TLSProtocolSend (+implementation) ============================================================================= IO Bug Fixes ]=============================================================== ============================================================================= [*] IOProcessor::SubmitIOWorkItem should signal the CvEvent, forcing at least once future tick (wont optimize with if in tick & not yet dispatched work items) [*] Split IOPipeWork in into IOPipeProcessor header [+] IOPipeWork::GetBuffer (internal reallocation) [*] Harden against IAsyncTransactions without a loop source [*] Missing null `if (processor->listener)` in IOProcessor [*] Solved some soft-lock conditions under Linux's LoopQueue (added deferred commits) [*] Quick hack: IOProcessor::HasItems() should OR the early can-tick check function. ============================================================================= Other ]====================================================================== ============================================================================= [+] Linux: LSSignalCatcher [+] `static void AuResetMember(Aurora::Memory::ByteBuffer &ref)` for AuROXTL [*] Attempt to enforce a normalization and don't overwrite-readptr-under-istreamwriters policy in ByteBuffer_ReadWrite (circular buffers) [*] Bad ECC ctors ============================================================================= Known issues ]=============================================================== ============================================================================= > Linux net is nowhere near done > UDP socket emulation layer isn't implemented > Ciphersuite API is a stub > Private key API is a stub > ...therefore no TLS servers > Missing thread safety precautions under net > Net implementation is still beri early
2022-08-28 19:02:06 +00:00
bool SourceAddWithTimeoutEx(const AuSPtr<ILoopSource> &source, AuUInt32 ms) ;
2022-04-05 05:59:23 +00:00
bool SourceRemove(const AuSPtr<ILoopSource> &source) override;
AuUInt32 GetSourceCount() override;
virtual bool AddCallback(const AuSPtr<ILoopSource> &source, const AuSPtr<ILoopSourceSubscriber> &subscriber) override;
virtual bool AddCallbackEx(const AuSPtr<ILoopSource> &source, const AuSPtr<ILoopSourceSubscriberEx> &subscriber) override;
virtual bool AddCallback(const AuSPtr<ILoopSourceSubscriber> &subscriber) override;
void ChugPathConfigure(AuUInt32 sectionTickTime, AuSInt sectionDequeCount) override;
void ChugHint(bool value) override;
virtual bool Commit() override;
2022-04-10 15:59:39 +00:00
bool IsSignaledPeek() override;
AuUInt32 PumpNonblocking() override;
AuList<AuSPtr<ILoopSource>> PumpNonblockingEx() override;
2022-04-05 05:59:23 +00:00
bool WaitAll(AuUInt32 timeout) override;
AuUInt32 WaitAny(AuUInt32 timeout) override;
AuList<AuSPtr<ILoopSource>> WaitAnyEx(AuUInt32 timeout) override;
2022-04-10 15:59:39 +00:00
AuUInt32 DoTick(AuUInt64, AuList<AuSPtr<ILoopSource>> *optOut = nullptr, bool *tryAgain = nullptr, bool async = false);
2022-04-05 05:59:23 +00:00
private:
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
void PumpHooks();
bool CommitDecommit();
struct SourceExtended
{
SourceExtended(LoopQueue *parent, const AuSPtr<ILoopSource> &source);
~SourceExtended();
void Deinit();
void Commit(const AuSPtr<SourceExtended> &self);
AuSPtr<ILoopSource> source;
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
ILoopSourceEx *sourceExtended {};
LoopQueue *parent {};
AuWPtr<SourceExtended> pin;
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
AuUInt64 timeoutAbs {};
bool ConsiderTimeout(AuUInt64 time) const
{
if ((timeoutAbs) && (time >= timeoutAbs))
{
for (const auto &handler : subscriberExs)
{
try
{
handler->OnTimeout(source);
}
catch (...)
{
SysPushErrorCatch();
}
}
return true;
}
return false;
}
AuList<AuSPtr<ILoopSourceSubscriber>> subscribers;
AuList<AuSPtr<ILoopSourceSubscriberEx>> subscriberExs;
bool bHasCommited {};
// ticked, should remove
AuTuple<bool, bool, bool> DoWork(int fd);
AuTuple<bool, bool, bool> DoWork(bool read, bool write);
};
struct AnEpoll
{
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
LoopQueue *parent {};
AuThreadPrimitives::SpinLock lock;
AuBST<int, int> startingWorkRead;
AuBST<int, int> startingWorkWrite;
void Add(SourceExtended *source);
void Remove(SourceExtended *source, bool readData, bool writeData);
};
int epollFd_{ -1 };
LSEvent lockStealer_;
AuThreadPrimitives::SpinLock commitQueueMutex_;
AuList<AuTuple<AuSPtr<ILoopSource>, AuSPtr<ILoopSourceSubscriber>, AuSPtr<ILoopSourceSubscriberEx>>> commitPending_;
AuList<AuSPtr<ILoopSource>> decommitQueue_;
AuThreadPrimitives::SpinLock globalLockMutex_;
AuList<AuSPtr<ILoopSourceSubscriber>> allSubscribers_;
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
AuThreadPrimitives::RWLockUnique_t sourceMutex_;
AuList<AuSPtr<SourceExtended>> sources_;
Further Linux support [+] Begin work on IO futexes for io release on process/thread exit [+] Linux ::readdir iteration [+] AuConsole buffering API [*] Fix sleep as to not get interrupted by signals [*] Switch the type of FS lock used under Linux [*] Linux: Use new IPCHandle encoding scheme [*] Fix undefined behaviour: unintialized timeout values (AuLoop/Linux) [*] Fix undefined behaviour: ConsoleTTY clear line was called of a color of a random value on stack [-] Remainings of std dir iterator [*] Fix pthread_kill (aka send signal to pthread handle) always kills process. This is what you expect bc signal handler inheritance. [*] Reformat the build Aurora.json file [+] Added clang warning ignores to the build file [*] Fix: UNIX need to use STDOUT_FILENO. Was using CRT handle in place of fd by mistake. [+] Linux implementation for IO yield (AuIO::IOYield() - UNIX::LinuxOverlappedYield()) [*] Fix: Linux async end of stream processing. res 0 = zero bytes consumed. <= was detecting this as an error of code 0. Should succeed with zero bytes. [+] Linux LoopQueue missing epilogue hook for the IO processor [*] Various refactors and minor bug fixes [*] Linux fix: Handle pipe EOS as zero [*] Linux fix: thread termination via a user signal of 77. Need a force terminate. [*] IPC handle: fix improper int to bool cast in the header setup within ToString [*] Linux fix: HWInfo CPU topology regression [-] Linux fix: remove SIGABRT handler [*] Missing override in compression, exit, and consoletty headers. [+] Unix Syslog logger backend
2022-08-02 04:52:17 +00:00
AuThreadPrimitives::RWLockUnique_t polledItemsMutex_;
AuList<AnEpoll *> alternativeEpolls_;
AuList<AuFunction<void()>> epilogueHooks_;
[+] Network + Protocol + TLS - Initial Commit ============================================================================= Network ]==================================================================== ============================================================================= [+] Added (very) early Aurora::IO::Net implementation [+] AuNet::EHostnameType [+] AuNet::EIPProtocol [+] AuNet::ENetworkError [+] AuNet::ETransportProtocol [+] AuNet::INetInterface [+] AuNet::INetSrvDatagram [+] AuNet::INetSrvResolve [+] AuNet::INetSrvSockets [+] AuNet::INetSrvWorkers [+] AuNet::INetWorker [+] AuNet::IPAddress [+] AuNet::IResolver [+] AuNet::ISocket [+] AuNet::IResolver [+] AuNet::ISocketBase [+] AuNet::ISocketChannel [+] AuNet::ISocketDriver [+] AuNet::ISocketDriverFactory [+] AuNet::ISocketServer [+] AuNet::ISocketServerDriver [+] AuNet::NetEndpoint [+] AuNet::NetError [+] AuNet::NetHostname (+implementation) ============================================================================= Protocol ]=================================================================== ============================================================================= [+] IProtocolInterceptor [+] IProtocolInterceptorEx [+] IProtocolStack (+implementation) ============================================================================= TLS ]======================================================================== ============================================================================= [+] ITLSContext [+] TLSProtocolRecv [+] TLSProtocolSend (+implementation) ============================================================================= IO Bug Fixes ]=============================================================== ============================================================================= [*] IOProcessor::SubmitIOWorkItem should signal the CvEvent, forcing at least once future tick (wont optimize with if in tick & not yet dispatched work items) [*] Split IOPipeWork in into IOPipeProcessor header [+] IOPipeWork::GetBuffer (internal reallocation) [*] Harden against IAsyncTransactions without a loop source [*] Missing null `if (processor->listener)` in IOProcessor [*] Solved some soft-lock conditions under Linux's LoopQueue (added deferred commits) [*] Quick hack: IOProcessor::HasItems() should OR the early can-tick check function. ============================================================================= Other ]====================================================================== ============================================================================= [+] Linux: LSSignalCatcher [+] `static void AuResetMember(Aurora::Memory::ByteBuffer &ref)` for AuROXTL [*] Attempt to enforce a normalization and don't overwrite-readptr-under-istreamwriters policy in ByteBuffer_ReadWrite (circular buffers) [*] Bad ECC ctors ============================================================================= Known issues ]=============================================================== ============================================================================= > Linux net is nowhere near done > UDP socket emulation layer isn't implemented > Ciphersuite API is a stub > Private key API is a stub > ...therefore no TLS servers > Missing thread safety precautions under net > Net implementation is still beri early
2022-08-28 19:02:06 +00:00
AuList<AuPair<AuSPtr<ILoopSource>, AuUInt32>> pendingBlocking_;
bool bRecommitLater {};
AnEpoll globalEpoll_;
2022-04-05 05:59:23 +00:00
};
}